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Abstract 

One  of  the  most  common  problems  in  Operations  Research  is  the  assignment 
problem.  It  deals  with  the  optimization  of  a  decision  makers’  goal  by  matching  objects  in 
one  group  (jobs)  with  objects  in  another  (machines).  Flight  scheduling  in  fighter 
squadrons  is  a  hard  and  complicated  problem  which  comes  with  a  dynamic  environment 
and  multiple  decision  makers  and  goals.  Using  pilots  as  machines  and  missions  to  be 
flown  as  jobs,  the  fighter  squadron  flight  scheduling  problem  can  be  solved  like  an 
assignment  problem  with  multiple  goals.  This  research  develops  a  new  way  to  solve  the 
multi-objective  assignment  problem  and  demonstrates  this  new  approach  using  the  fighter 
squadron  flight  scheduling  problem  as  an  example. 

In  this  research,  the  Value  Focused  Thinking  method  is  applied  to  build  a  decision 
analysis  model  to  help  decision  makers  in  fighter  squadrons  evaluate  the  mission-pilot 
matches.  The  decision  model  built  with  the  help  of  experienced  schedulers  is  used  not 
only  for  evaluating  matches  but  also  for  ordering  assignments  to  see  priorities.  To  verify 
and  validate  this  model,  ten  groups  of  alternatives  were  randomly  created  and  evaluated 
by  the  model  and  the  decision  maker.  The  results  from  this  analysis  show  that  the 
decision  model  is  valid  and  proved  to  be  helpful  and  accelerated  the  assignment  matching 
process. 
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MULTI  OBJECTIVE  DECISION  ANALYSIS  FOR  ASSIGNMENT  PROBLEMS 


1.  INTRODUCTION 

1.1  General  Problem 

In  all  organizations,  the  scheduling  process  plays  an  important  role.  The  scarcity 
of  time,  resources,  money  and  the  abundance  of  tasks  has  the  process  coping  with  a  huge 
problem.  Tradeoffs  between  objectives  are  also  difficult  for  decision  makers  in  these 
kinds  of  problems.  To  sum  up,  “Scheduling  is  a  decision-making  process  that  is  used  on  a 
regular  basis  in  many  manufacturing  and  services  industries.  It  deals  with  the  allocation 
of  resources  to  tasks  over  given  time  periods  and  its  goal  is  to  optimize  one  or  more 
objectives”  (Pinedo,  2008) 

In  scheduling  problems,  resources  are  named  machines  and  tasks  are  named  jobs. 
Schedulers  are  responsible  with  finding  the  sequence  of  jobs  which  is  compatible  with 
machine  and  time  constraints  and  optimal  with  respect  to  multiple  objectives.  The 
objectives  have  various  types,  like  minimizing  total  completion  time  or  minimizing 
tardiness. 

One  way  of  solving  these  types  of  problems  is  to  slightly  modify  the 
transportation  problem.  Operations  Researchers  call  this  problem  an  assignment 
problem.  In  this  problem,  there  are  a  certain  numbers  of  jobs  and  machines  and  the 
decision  maker  wants  to  optimize  his/her  goals  by  a  well  matched  job -machine 
assignment.  Chapter  2  discusses  assignment  problems  in  detail. 
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1.2  Specific  Problem 


Hight  scheduling  in  fighter  squadrons  can  be  seen  as  an  assignment  problem  by 
thinking  of  machines  as  pilots  and  jobs  as  missions.  Unlike  most  scheduling  problems, 
sequeneing  the  flights  is  not  as  important  as  the  mission-pilot  assignment.  The  most 
important  process  is  matehing  jobs  and  maehines  aecording  to  constraints  and  optimizing 
the  objectives.  In  Chapter  3,  the  objeetives  and  eonstraints  of  fighter  squadron  flight 
seheduling  are  diseussed. 


1.3  Scope  of  the  Research 

This  research  focuses  on  evaluating  the  assignments  of  pilots  to  missions  in  a 
specific  time  period  which  can  be  a  block  of  the  day  or  a  full  day.  In  this  technological 
era  of  human  life,  having  pilots  schedule  flights  versus  flying  themselves  is  wasting  time 
and  resources  which  are  invaluable.  If  the  scheduler  could  be  replaced  with  a  model  or 
have  their  job  automated  or  made  easier,  it  would  be  a  great  time  saver  and  more  reliable. 
This  research  uses  Value  Focused  Thinking  (VFT)  with  the  help  of  responsible  decision 
makers  to  develop  evaluation  criteria  to  do  just  that.  With  this  method,  a  model  is  made, 
and  this  model  is  used  to  determine  new  alternatives  (pilot-mission  assignments).  In 
Chapter  2,  the  VFT  method  is  discussed  in  more  detail.  Briefly,  the  process  of  modeling 
the  scheduler  has  three  steps: 

i.  Building  an  evaluation  model  using  VFT  (Defining  objectives  and  values). 

ii.  Using  the  evaluation  model  structure  to  aid  the  scheduler  in  manually 
building  schedules  (Decision  Support  System). 
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iii.  Automating  the  process  of  pilot-mission  assignment  with  the  help  of 
defined  values  and  objectives. 

The  scope  of  this  research  is  the  first  two  steps  of  modeling  the  scheduler’s 
process  and  the  third  step  is  left  as  future  research. 

1.4  Research  Objectives 

First,  this  research  provides  simplicity,  flexibility  and  structured  thinking  to 
assignment  problems.  Specifically,  it  structures  the  preferences  of  decision  makers  and 
starts  to  build  assignments  to  meet  the  decision  makers’  objectives.  So,  it  is  a  value  and 
objective  based  solution  for  assignments.  Second,  it  shows  the  big  picture  of  the  current 
state  of  the  problem  to  decision  makers.  The  third  goal  is  to  change  subjective  decisions 
to  objective  ones.  Given  an  objective  model,  the  decisions  will  be  repeatable  and 
consistent.  Finally,  the  last  goal  is  to  reduce  the  total  time  it  takes  a  scheduler  to  do  their 
job.  Thus  the  research  question  is: 

How  can  an  objective  assignment  matching  process  be  facilitated  and  accelerated 
by  a  flexible  computer  model? 


1.5  Summary 

In  this  chapter,  the  general  and  specific  problem  of  this  research  was  discussed. 
Also,  the  scope  and  objective  of  the  research  are  stated.  In  Chapter  2,  the  background  of 
assignment  problems,  recent  research  about  thesis  problems,  previous  researches  in  flight 
scheduling,  decision  analysis  in  flight  scheduling  and  the  VFT  method  are  discussed.  The 
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methodology  is  held  in  Chapter  3  and  it  is  analyzed  in  Chapter  4.  Finally,  in  Chapter  5, 
conclusions  are  presented. 
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2.  LITERATURE  REVIEW: 


This  section  states  a  brief  background  of  the  assignment  matching  problem,  followed 
by  previous  research.  Then,  it  discusses  previous  research  in  the  flight  scheduling  area. 
Finally,  the  Value  Focused  Thinking  (VFT)  method  is  discussed. 

2.1  Background 

“The  assignment  matching  problem  is  a  well  known  combinatorial  optimization 
problem  in  the  field  of  Operations  Research  (OR).  There  is  a  plethora  of  archived 
research  for  the  assignment  matching  problem  such  as  the  generalized  assignment 
problem,  traffic  assignment  problem,  quadratic  assignment  problem,  and  the  job 
assignment  problem”  (Kleeman,  2007).  The  assignment  problem  is  basically  a 
transportation  problem.  There  is,  however,  a  slight  difference  between  them.  “The 
assignment  matching  problem  is  generally  described  as  assigning  a  number  of  elements 
(e.g.,  people  or  machines)  to  a  number  of  positions  (e.g.,  jobs  or  tasks)  with  the  goal  of 
assigning  all  elements  to  positions  given  a  certain  cost  element”  (Kleeman,  2007).  The 
transportation  problem  is  basically  aiming  to  minimize  the  cost  of  shipping  goods  from  one 
location  to  another  so  that  the  demands  of  every  depot  or  shop  are  met  and  every  shipping 
location  works  within  its  capacity. 

Modeling  assignment  matching  problems  using  bipartite  graphs  is  generally  preferred 
because  they  are  well  known  problems  for  which  many  polynomial  algorithms  exist  (i.e., 
many  problems  are  either  a  form  of  an  assignment  matching  problem  or  an  extension  of  an 
assignment  matching  problem  with  additional  constraints)  (Caseau,  2000). 
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Figure  1 :  Bipartite  Graph  Presentation  (Caseau,  2000) 

In  Figure  1,  an  example  of  a  bipartite  graph  is  shown.  The  resources  are  on  the 
left  and  the  jobs  are  on  the  right.  The  arrows  define  constraints  and  also  the  cost  of  the 
match. 


2.1.1  Recent  Research  in  Assignment  Problem 

In  Jeong’s  research,  he  assigns  US  Air  Force  new  recruits  to  available 
jobs.  The  goal  is  to  find  the  best  assignments  in  an  efficient  manner.  He  modeled  this 
problem  as  a  bipartite  assignment  matching  problem.  He  uses  two  multi-criteria 
optimization  techniques,  lexicographic  optimization  and  the  elastic  constraint  method,  the 
assignment  matching  algorithm  efficiently  produces  an  optimal  solution  in  a  fraction  of 
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the  time  currently  spent  (Jeong,  2010).  He  uses  single-objective  approach  for  sub 
problems  but  might  use  multi-objective  approach  to  get  better  results  from  them.  After 
all,  his  model  saves  significant  time  for  users  and  meets  their  goals  with  high 
percentages. 

2.1.2  Recent  Researches  in  Flight  Scheduling 

In  this  section  two  different  methods  for  solving  the  flight  scheduling 
problem  are  reviewed. 

Newlon  introduces  the  fighter  squadron  scheduling  problem  where  the 
solution  method  is  a  mathematical  model.  As  weekly  schedules  are  tedious  and  can  take  a 
long  time  to  complete,  using  new  technology  and  improvements  in  mathematical  models 
can  be  profitable  in  any  squadrons’  time  consuming  problems  as  well.  The  main  concern 
in  his  research  is  building  a  feasible  schedule  if  possible.  Some  of  the  requirements  that 
restrict  the  schedules  are  crew  rest,  days  since  a  pilot’s  last  sortie,  sorties  in  the  last  30 
days,  and  sorties  in  the  last  90  days. 

Newlon  uses  two  methodologies  to  solve  the  problem  and  then  compares 
them.  The  first  methodology  separates  a  week  into  ten  blocks  and  calls  them  sub¬ 
problems.  After  getting  the  sub-problems,  it  starts  to  solve  problems  to  optimality  within 
the  given  constraints.  The  first  technique  makes  updates  between  blocks  of  the  week.  The 
second  methodology  considers  problems  as  a  weekly  schedule  and  does  not  separate  the 
week  into  the  parts.  It  solves  the  weekly  schedule  to  optimality  within  the  given 
constraints.  This  technique  has  a  disadvantage  of  not  being  able  to  update  needed  pilot 
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data  and  other  information  changed  from  the  first  day  to  the  last  day  of  the  week. 
Nevertheless,  it  takes  advantage  of  optimizing  the  problem. 


Newlon  considers  the  problem  as  purely  a  scheduling  problem  and  solves 
it  to  optimality  with  the  help  of  scheduling  methods  and  formulas.  The  model  defines 
certain  number  of  hours  to  use  for  flying,  briefing  and  debriefing,  other  duties,  resting 
and  other  work.  It  limits  the  time  of  day  allotted  for  flying  and  resting.  Then  the  program 
starts  to  place  the  matching  of  pilot-mission  (duty  or  etc.)  to  the  respective  cells  in  the 
schedule.  Finishing  up  the  necessary  placements,  the  model  sometimes  needs  a  generic 
pilot  to  fly  for  the  optimization  of  the  block  schedule  or  weekly  schedule.  That  means 
somebody  outside  of  the  squadron  will  be  called  to  fly  for  the  squadron.  Newlon’ s 
program  penalizes  this  kind  of  gap  in  the  model  and  tries  to  handle  the  scheduling 
problem  with  the  help  of  the  squadron’s  own  resources. 

Newlon’ s  research  also  scopes  the  construction  part  of  the  scheduling 
problem.  As  a  scheduling  problem,  the  solution  model  is  fast  enough  to  get  the  attention 
of  responsible  people  in  the  squadrons.  It  solves  the  problem  to  optimality  in  seconds. 
However,  the  research  mentions  that  the  objective  function  is  rough  and  that  more 
importance  may  need  to  be  paid  to  the  objectives  instead  of  the  speed  of  the  model 
(Newlon,  2007). 

Yavuz’s  research  is  also  about  automating  the  weekly  flight  schedule  for 
fighter  squadrons;  however,  his  research  is  focused  on  Turkish  F-16  squadrons.  First,  he 
starts  by  getting  grades  for  pilot-mission  matchings.  For  calculating  these  grades,  he 
defines  some  inputs  and  formulas  in  the  methodology  part.  Secondly,  he  chooses  GRASP 
as  a  heuristic  method  because  the  problem  is  NP  hard  and  the  environment  of  the 
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scheduling  section  is  dynamic.  Even  the  grades  are  not  known  with  complete  confidence; 
schedulers  do  not  use  a  fixed  objective  function  to  get  optimal  schedules.  Also,  some 
other  decision  makers  affect  the  weekly  scheduling  problem  and  having  more  than  one 
decision  maker  extends  the  process  too  much.  He  could  use  a  Linear  Program  (LP)  but  it 
would  take  too  much  time  to  get  an  optimal  solution  from  the  program  due  to  lots  of 
constraints  and  variables.  Besides,  after  getting  an  optimal  solution  from  this  program, 
there  might  still  be  disagreement  that  it  is  suitable  in  real  life.  Due  to  these  reasons,  using 
a  heuristic  and  consuming  minimal  time  is  a  perfect  choice  for  that  kind  of  problem. 

Yavuz  describes  two  main  objectives  for  the  schedulers.  The  first  one  is 
mission  currency  which  means  a  limitation  for  a  pilot  to  fly  a  certain  type  of  mission  in  a 
certain  number  of  days  to  be  able  to  fly  solo  again  in  that  mission.  The  second  one  is  the 
number  of  flights  by  every  pilot  in  the  corresponding  month.  Using  the  automated  tool, 
he  aims  to  make  all  the  pilots  current  on  every  mission  and  schedule  them  equitably  in 
the  corresponding  month. 

This  research  has  strong  points  on  its  speed  and  updating  process.  The 
program  is  written  with  MATLAB,  provides  a  weekly  program  in  seconds,  and  is 
incredibly  faster  than  a  human  scheduler.  Contrary  to  past  researcher’s  codes  in  that 
subject,  Yavuz’ s  code  updates  matrices  related  to  information  concerning  pilots  and 
missions  after  all  blocks  of  a  day.  However,  the  research  emphasizes  only  the  last  step  of 
the  schedule  process  and  it  does  not  pay  attention  to  the  grading  part  which  decides  the 
objectives  of  schedule.  One  problem  with  the  research  is  schedulers  in  squadrons  are  not 
interested  in  programs  written  with  MATLAB;  they  always  seem  to  be  interested  in 
visual  aided  programs  that  are  more  understandable  and  flexible  for  them  (Yavuz,  2010). 
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2.2  Decision  Analysis  in  Flight  Scheduling 


In  this  research,  chapter  one  and  previous  parts  of  chapter  two  talk  about 
the  main  problems  and  a  more  specific  problem.  The  main  problems  of  this  research  in 
the  Operation  Research  arena  are  the  scheduling  and  assignment  problems.  There  are 
many  methods  written  and  created  by  researchers  for  approaching  these  kinds  of 
problems.  One  of  them  is  reviewed  in  detail  in  this  section.  The  specific  problem  of  this 
research  is  flight  scheduling  in  fighter  squadrons.  Many  researchers  in  this  area  think  it  is 
a  scheduling  problem  and  apply  some  scheduling  methods  to  solve  the  problem  to 
optimality.  If  we  consider  the  time  blocks  in  squadrons’  schedules,  we  also  can  think  of  it 
as  an  assignment  problem.  It  is  simply  a  pilot -mission  matching  problem  instead  of  using 
some  updates  while  time  is  passing. 

Assignment  problems  have  some  weights  or  scores  for  a  matching  and 
some  constraints  concerning  which  matchings  can  occur.  Flight  scheduling  in  fighter 
squadrons  has  priorities  about  pilot-mission  matches  in  a  certain  block  of  the  day.  But  it 
is  essential  that  the  priorities  are  able  to  change  from  one  block  of  day  to  the  others.  Thus 
the  problem  is  not  a  static  problem;  it  is  extremely  dynamic.  Schedulers  have  major 
problems  deciding  the  priorities  between  pilot-mission  matches.  Some  commanders  and 
directors  in  the  squadrons  name  the  objectives  themselves  while  deciding  the  matches 
and  also  have  written  objectives,  limitations  and  constraints.  Finally  we  can  define  the 
specific  problem  in  squadrons  as  a  multi  objective  decision  problem. 

In  decision  analysis,  there  are  two  major  methods  of  thinking.  The  first 
one  is  Alternative  Focused  Thinking  (AFT)  and  it  uses  alternatives  as  the  basis  of  the 
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decision.  A  project  manager  gets  the  alternatives  and  investigates  them  for  their  strengths 
and  weaknesses.  After  the  investigation,  he  or  she  presents  the  results  of  the  research  and 
tries  to  help  DMs  think  about  the  problem.  The  second  method  is  Value  Focused 
Thinking  (VFT)  and  this  method  starts  with  building  a  structure  for  the  values  of  the  DM 
and  organization.  It  combines  everybody’s  preferences  on  the  subject  while  constmcting 
the  model.  After  finishing  with  the  model  structure,  the  alternatives  can  be  ranked  via  this 
model.  The  most  significant  difference  between  these  methods  is  the  starting  point.  AFT 
starts  by  searching  for  alternatives  and  VFT  begins  with  searching  for  values  which  are 
important  to  DMs. 

Fighter  squadrons  are  the  core  unit  of  Air  Forces  and  they  have  strict 
discipline  while  doing  their  tasks.  So,  their  decisions  about  schedules  must  also  have 
principles  and  strict  guidelines  that  are  not  changing  because  of  relocating  people  in  the 
squadron.  Due  to  the  need  for  a  robust  decision  structure,  VFT  serves  best  in  this  sort  of 
circumstance.  Commanders  and  directors  of  operation  and  training  sections  in  the 
squadron  can  build  a  value  model  structure  and  evaluate  the  schedules  by  the  structure  to 
decide  on  daily  or  weekly  flight  schedules. 

2.3  Value-Focused  Thinking  (VFT) 

In  this  research,  pilot-mission-time  matching  is  evaluated  through  the  use 
of  Value-Focused  Thinking  (VFT).  VFT  is  a  “structured  method  for  incorporating  the 
information,  opinions,  and  preferences  of  the  various  relevant  people  into  the  decision 
making  process”  (Kirkwood,  1997).  VFT  is  a  strategic,  quantitative  approach  to  decision 
making  that  relies  on  specified  objectives,  evaluation  considerations,  evaluation 
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measures,  and  value  hierarchies  (Kirkwood,  1997).  Values  are  defined  as  the  issues  that 
are  important  to  the  decision  maker.  The  VFT  process  is  a  sequence  of  five  activities: 
recognize  a  decision  problem,  specify  values,  create  alternatives,  evaluate  alternatives, 
and  select  an  alternative  (Keeney,  1992).  Thinking  about  values  first  has  some 
advantages. 
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Figure  2:  Benefits  of  Vaiue  Focused  Thinking  (Keeney,  1992) 


The  figure  above  mentions  some  of  the  advantages  that  come  with  the  VFT  method. 
By  using  this  method  the  alternative  that  has  the  most  value  can  be  seen  or  it  can  be  created 
using  the  stated  values. 

Shoviak’s  more  detailed  VFT  method  uses  ten  steps:  identifying  a  problem, 
creating  a  value  hierarchy,  developing  evaluation  measures,  creating  single  dimensional 
value  functions,  weighting  the  value  hierarchy,  generating  alternatives,  scoring  the 
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alternatives,  conducting  deterministic  analysis,  conducting  sensitivity  analysis,  and  providing 
conclusions  and  recommendations.  (Shoviak,  2001). 


Figure  3  :  Value-Focused  Thinking  10  Step  Process  (Shoviak,  2001) 


2.3.1  Problem  Identification 

The  process  starts  with  identifying  the  problem.  The  major  goals  in  this  step  are 
identifying  the  right  problem  and  identifying  it  correctly.  Although  the  process  begins 
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with  a  known  problem  at  hand,  it  is  more  important  to  define  fundamental  issues 
correctly  and  completely.  If  the  right  problem  is  identified  and  decision  makers  agree 
with  that,  then  precious  resources  will  not  be  wasted. 

2.3.2  Create  Value  Hierarchy 

After  defining  the  preferences,  objectives  and  values,  this  step  structures  them 
hierarchically.  A  “value  structure  encompasses  the  entire  set  of  evaluation  considerations, 
objectives,  and  evaluation  measures  for  a  particular  decision  analysis”  (Kirkwood,  1997). 
The  basic  questions  for  this  step  are  “What  is  important?”  and  “What  do  we  value?”.  The 
first  states  every  possible  concern  as  evaluation  consideration;  the  second  defines  the  desired 
direction  for  evaluation  considerations  as  objectives  and  finally  assesses  them  by  a  scale  and 
is  called  an  evaluation  measure.  Kirkwood  says  that  when  a  value  structure  is  organized 
hierarchically,  the  structure  is  called  a  value  hierarchy  (Kirkwood,  1997). 
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Figure  4:  Example  of  a  Generic  Value  Hierarchy  (Jeoun,  2005). 


The  figure  above  shows  the  generic  version  of  a  value  hierarchy.  In  this 
structure  the  first  box  is  the  main  question  or  the  main  problem.  After  revealing  the  top 
box,  the  rest  of  the  structure  consists  of  tiers  and  branches.  A  tier  consists  of  all  the 
evaluation  considerations  that  are  the  same  distance  from  the  top  of  the  hierarchy  (Kirkwood, 
1997).  Branches  are  composed  of  all  the  objectives  and  evaluation  measures  that  derive  from 
a  single  evaluation  consideration  (Bulson,  2006). 

A  value  hierarchy  has  five  desirable  properties:  completeness,  non  redundancy, 
independence,  operability,  and  small  size  (Kirkwood,  1997).  Completeness  means 
covering  all  evaluation  concerns  for  the  problem.  In  a  non-redundant  structure,  evaluation 
considerations  should  not  overlap.  Completeness  and  non-redundancy  is  often  called 


24 


collectively  exhaustive  and  mutually  exclusive.  Independence  in  a  VFT  structure  is  defined 
as  preferential  independence.  If  a  decision  maker’s  preference  for  the  level  of  one  evaluation 
measure  does  not  depend  on  the  level  of  the  other  evaluation  measure,  the  value  hierarchy  is 
preferentially  independent.  Operability  and  small  size  properties  facilitate  each  other.  They 
are  for  increasing  understanding  and  usability  by  decision  makers  or  subject  matter  experts. 

2.3.3  Develop  Evaluation  Measures 

The  next  step  in  the  process  of  building  a  VFT  structure  comes  with  the 
question  “How  can  we  evaluate  what  we  value?”.  Defining  objectives  and  values  in  the 
first  tiers,  the  last  tier  of  the  hierarchy  takes  place  with  the  measures  that  evaluate  the 
objectives  and  values  above  them.  For  instance,  if  we  choose  income  as  a  value  in  our 
structure,  we  could  select  annual  income  of  the  company/family  in  dollars  as  an 
evaluation  measure  of  this  value  at  the  last  tier  under  the  branch  of  income.  After 
defining  the  name  and  the  unit  of  the  measure,  we  have  to  state  the  range  of  it.  Again  in 
our  example,  we  could  determine  the  range  of  annual  income  in  dollars  as  zero  to  one 
million.  This  range  is  important  to  specify  because  later  steps  will  apply  it  to  some 
formulas  and  some  graphs  for  obtaining  the  final  scores  of  the  specific  objective 
evaluation. 

2.3.4  Create  Value  Functions 

In  order  to  evaluate  values,  we  attain  measures  in  the  previous  step  but  we 
need  to  acquire  some  formulas  or  graphs  to  convert  measures  to  numbers.  Furthermore, 
because  the  structure  possesses  several  values  to  evaluate  and  compare  them  to  each 
other,  we  also  desire  to  get  normalized  scores  from  all  individual  values.  VFT  answers 
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these  questions  by  Single  Dimensional  Value  Functions  (SDVF).  All  values  are  evaluated 
via  their  SDVF’s  and  finalized  with  zero  to  one  ranged  scores. 

Considering  the  values  of  the  problem  we  do  not  always  have  quantitative 
measured  values.  At  this  point,  the  SDVF  method  helps.  By  this  method,  the  DM  or  SME 
can  be  questioned  and  a  function  with  zero  to  one  ranged  scores  can  be  achieved.  For 
example,  we  could  choose  motivation  of  the  students  in  a  primary  school  as  a  value  in 
our  hierarchy  and  the  measure  could  be  happiness  level.  It  seems  to  be  a  tough  value  to 
evaluate  by  formulas  or  quantitative  methods.  But  building  a  SDVF  by  questioning 
teachers  and  students  and  their  observation  of  happiness,  we  could  have  a  score  that 
ranges  zero  to  one  and  use  it  in  our  model. 

2.3.5  Weight  Value  Hierarchy 

Most  of  the  decision  problems  are  multi  objective  decision  problems  and 
there  are  priorities  between  these  objectives.  In  the  VFT  structure,  if  the  objectives  were 
equally  important,  we  would  add  the  scores  of  all  branches  and  finish  up  with  a  final 
score.  However,  real  life  makes  the  situations  more  complex  and  humans  compare  and 
prioritize  objectives  in  their  minds.  Thus  we  cannot  simply  assume  equal  importance. 
These  prioritizations  generate  the  next  step  in  VFT.  This  is  called  the  weighting  process. 
There  are  two  general  ways  of  weighting  a  value  hierarchy.  The  first  one  starts  from  the 
bottom  and  is  called  global  weighting.  At  the  bottom  of  the  structure,  the  DM  is  asked  to 
consider  the  differences  of  values  and  weight  them  for  the  purpose  of  arranging  a 
preference  order.  In  spite  of  the  advantage  of  seeing  all  the  values  and  having  a 
preference  order,  this  method  can  be  tedious  when  we  are  weighting  a  large  structure.  At 
this  point,  the  second  method  of  weighting  is  useful  and  is  easier  to  apply.  It  is  called 
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local  weighting  and  it  starts  from  the  top  of  the  structure.  In  this  method,  the  DM  is  asked 
to  order  preferences  by  branches.  The  most  challenging  disadvantage  in  this  method  is 
inconsistencies  between  branches  can  occur  in  some  models.  This  can  be  corrected  by 
calculating  and  inspecting  the  global  weightings  after  finishing  with  the  local  weighting 
process.  Another  difference  between  methods  is  global  weighting  where  the  sums  of  the 
bottom  tiers’  weights  sum  to  one  but  on  the  other  hand  local  method  sums  each  branches’ 
weights  to  one. 

2.3.6  Alternative  Generation 

The  previous  steps  introduce  the  framework  to  the  DM  for  thinking  about 
the  problem  comprehensively.  In  this  step,  alternative  solutions  to  the  problem  must  be 
generated.  Alternatives  can  be  present  or  waiting  for  discovery.  Sometimes  there  will  be 
large  number  of  alternatives  to  be  scored;  because  of  this,  the  DM  might  want  to 
eliminate  some  of  the  related  alternatives.  Usually,  a  screening  method  works  well  in 
order  to  solve  problems  with  high  number  of  alternatives  but  it  is  essential  to  pay 
attention  while  deciding  screening  criteria.  Due  to  inaccurate  criteria,  the  model  can  miss 
good  alternatives  and  the  great  effort  made  for  structuring  the  model  might  be  wasted. 

The  second  type  of  problem  is  having  too  few  alternatives  to  solve  the 
problem.  Then,  VFT  presents  help  so  that  the  DM  can  use  the  values  that  he  or  she 
defined  before  and  can  develop  other  alternatives.  Certain  values  can  be  discussed  to 
maximize  their  scores  and  this  method  aids  the  DM  in  developing  smart  choices  as 
alternatives. 
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2.3.7  Alternative  Scoring 


This  step  brings  a  data  search  task  for  the  modeler.  Former  steps  state  the 
measures  and  alternatives,  and  then  the  model  needs  information  about  the  measures  of 
alternatives.  So  the  modeler  has  to  ask  DMs  or  SMEs  for  needed  data  or  try  to  get  data 
from  related  databases. 

2.3.8  Deterministic  Analysis 

In  order  to  determine  an  overall  score  for  each  alternative,  an  additive 
value  function  is  used.  The  form  of  this  function  is  a  weighted  sum  of  the  single 
dimensional  value  functions  over  each  evaluation  measure  (equation  shown  below) 
(Kirkwood,  1997). 

n 

v(x)  =  ^Wj  Vi  (Xi) 

i=l 

(1) 

Where: 

v(x)  =  overall  score  for  alternative  x 

Wi  =  global  weight  for  evaluation  measure  i 

Vi(xi)  =  value  score  for  alternative  x  from  SDVF  for  measure  i 

Xi  =  score  for  alternative  x  on  measure  i 

n  =  total  number  of  evaluation  measures 

Alternatives  can  be  shown  in  an  order  based  on  the  final  scores.  The  order 
is  constructed  with  the  weights  and  SDVFs  the  modeler  made  by  soliciting  the  DM  or 
SMEs.  Thus  we  cannot  say  the  order  is  certain  due  to  the  fact  that  we  created  the  model 
with  the  help  of  a  non-calibrated  DM. 
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23.9  Sensitivity  Analysis 


As  mentioned  in  the  previous  step,  a  deterministic  analysis  is  not  an 
absolute  and  reliable  analysis  for  a  decision.  Weights  and  value  functions  are  manmade 
and  humans  can  make  mistakes  while  building  the  model.  It  is  mostly  seen  that  humans 
have  overconfidence  when  they  are  solicited  for  probabilities  or  information  they  usually 
work  on.  Another  widely  seen  error  is  making  biased  decisions  so  that  we  cannot  be  sure 
about  the  numbers  we  acquired  from  DMs  or  SMEs. 

A  sensitivity  analysis  can  be  run  by  the  modeler  to  show  how  changes  in 
weights  affect  changes  in  final  scores.  Decision  makers  usually  have  certain  ranges  in 
mind  but  they  do  not  have  certain  numbers.  It  is  dangerous  to  say  this  branch’s  weight  is 
a  certain  number;  however,  we  can  ask  for  a  range  of  weights.  By  conducting  a 
sensitivity  analysis  on  this  range  of  weights,  the  modeler  can  bring  some  other  results  to 
light  and  make  DMs  think  again  about  their  choices  of  weights  and  the  final  decision. 

2.3.10  Conclusions  and  Recommendations 

This  step  contains  conclusions  and  recommendations.  The  modeler  can  list 
some  solutions  for  the  problem  by  the  aid  of  deterministic  and  sensitivity  analysis.  An 
important  nuance  the  modeler  must  pay  attention  to  is  that  using  these  decision  models  is 
not  making  decisions  for  DMs;  it  is  only  giving  them  a  picture  of  the  situation  from  an 
aspect  they  cannot  see  without  the  model. 
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3.  METHODOLOGY 


In  this  research,  a  new  approach  is  used  for  the  Air  Force  Fighter  Squadrons’  pilot 
scheduling  problem.  This  chapter  states  the  definitions  and  specifications  of  flight 
scheduling  in  fighter  squadrons  and  then  implements  VFT  for  modeling  the  solution  to 
this  problem. 

3.1  Flight  Scheduling  in  Fighter  Squadrons 

In  this  section,  the  general  information  about  flight  scheduling  in  fighter 
squadrons  will  be  revealed.  Also,  the  variables,  objectives  and  resources  of  the  problem 
will  be  explained. 

3.1.1  Flight  Schedules  in  Turkish  F-16  Squadrons 

In  F-16  Squadrons  of  the  Turkish  Air  Force,  there  are  approximately  five  people 
who  are  responsible  for  structuring  the  schedule.  One  of  them  is  a  director,  two  or  three 
of  them  are  planners  and  the  others  are  assistants.  The  planner  of  the  week  starts  to  make 
the  weekly/daily  schedule  with  the  help  of  assistants  and  computers.  After  having  an 
initial  schedule,  the  director  reviews  it  and  makes  changes  or  comments.  In  the  light  of 
this  information  the  planner  revises  the  initial  schedule.  This  procedure  continues  until 
the  director  approves  it.  However,  the  process  doesn’t  finish  here.  A  similar  period  is 
seen  after  this  approval  when  the  Squadron  Commander  (SC)  comes  into  play.  The 
schedule  is  only  final  after  the  SC  is  satisfied  with  it  and  signs  it  to  be  published. 

Considering  this  process,  the  general  problem  is  that  there  are  lots  of  hands  and 
eyes  on  the  schedule.  They  all  have  rights  to  change  it  and  add  their  opinions  according 
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to  their  objectives.  These  circumstances  make  this  operation  unstable,  subjective,  and 
challenging.  After  reviewing  the  process  and  naming  it  as  the  general  problem,  we  have 
the  specific  problem  which  is  the  schedule  itself.  The  planner  has  a  variety  of  inputs  and 
a  lot  of  alternatives  for  output.  Also,  there  are  constraints  that  may  change  during  the 
course  of  time.  The  planner  is  an  active  pilot  as  well.  He/she  makes  mission  planning, 
participates  in  mission  briefings,  flies  and  takes  part  in  debriefings. 

To  sum  up  we  have  one  human  brain  which  is  full  of  other  thoughts,  on  the  other 
hand  there  are  lots  of  variables,  constraints  and  decision  maker  inputs  to  consider  for  the 
scheduling  problem.  Beyond  all  that  is  a  time  constraint  to  finish  this  job. 

3.1.2  Flight  Schedule  Variables 

Normally,  schedules  have  two  or  three  time  blocks.  These  are  morning  (AM), 
afternoon  (PM)  and  night  blocks.  In  each  block  we  can  simply  see  mission  types,  plane 
numbers,  pilot  names,  area  names  and  times  of  events  (take  off,  landing  etc.).  Under  all 
information  of  blocks  we  also  have  duty  types  and  pilots  on  duty.  This  is  the  big  picture 
for  a  daily  flight  schedule. 

3.1.3  Missions 

Basically  there  are  two  types  of  missions.  First  are  operational  missions.  Those 
are  not  scheduled  by  squadrons.  They  are  assigned  from  headquarters  and  then  squadrons 
only  schedule  pilots  to  plan  and  fly  them.  Second  are  training  missions.  Those  are 
scheduled  by  squadron  planners.  There  is  a  variety  of  training  mission  types.  The  planner 
is  mainly  concerned  with  the  pilots  who  can  fly  the  mission  and  after  that  thinks  about  the 
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needs  of  the  plane  type  and  count.  At  this  point  we  meet  a  term  which  is  currency.  If  we 
say  this  pilot  can  fly  this  mission  with  a  one-seat  plane  this  means  the  pilot  is  current  on 
this  mission.  It  is  a  strict  constraint  for  the  planner  to  pay  attention  to  currencies  during 
flight  scheduling.  On  the  other  hand,  currency  shows  pilot’s  readiness  for  missions  and 
generally  for  fight.  Considering  squadrons’  goals,  we  may  name  currency  as  the  main 
objective  for  the  schedules. 

3.1.4  Planes 

In  F-16  squadrons  there  are  two  types  of  aircraft.  According  to  seat  count,  they 
are  named  the  F-16C  (single-seat)  or  F-16D  (double-seat).  As  we  learned  before  a  pilot 
which  is  current  for  a  mission  can  fly  with  a  single-seat  plane  on  this  mission.  But  if  the 
pilot  is  not  current  on  a  mission  or  he/she  needs  some  orientation,  he/she  can  fly  in  front 
cockpit  of  F-16D  with  an  instructor  seated  in  the  rear  cockpit.  Everybody  can  fly  any 
mission  in  rear  cockpit  of  F-16D  with  an  instructor  or  four-ship  leader  seated  in  the  front 
cockpit  without  concerning  currency  limitations. 

3.1.5  Pilots 

There  are  two  ways  to  divide  pilots  into  groups.  The  first  one  is  by  position  in  the 
squadron.  The  Squadron  Commander  is  the  number  one.  Operations  Officer  is  second 
and  Director  of  Flights  is  third.  After  them  we  have  positions  about  flights.  The  second 
way  to  divide  them  is  by  flight  experience.  The  pilots  who  are  most  experienced  and  have 
passed  the  instructors  course  are  called  instructors.  They  can  fly  in  the  rear  cockpit  to 
teach  new  pilots.  After  them,  four-ship  leaders  come  next  in  the  queue  of  experience. 
They  can  fly  with  a  four  ship  formation  in  the  flights  and  they  are  responsible  for  the 
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mission  all  the  time  starting  from  planning  until  debriefing  finishes.  Two-ship  leaders  can 
be  responsible  for  two  ship  formation  flights.  Other  than  leaders  there  are  wingmen  who 
are  flying  the  missions  but  they  are  not  responsible  for  the  others  like  leaders.  Instructors 
and  four-ship  leaders  can  fly  in  every  position  in  every  formation.  Two-ship  leaders  can 
fly  in  third  place  of  four-ship  missions,  first  place  of  two-ship  missions  and  also  as 
wingmen  for  any  mission.  Wingmen  are  allowed  to  fly  in  second  or  fourth  positions  of 
two  and  four-ship  missions. 

3.1.6  Objectives 

Every  pilot  has  to  fly  a  certain  number  of  flights  in  a  year.  Planners  divide  this 
number  into  months  and  want  to  obey  this  rule  to  have  all  pilots  equal  in  number  of 
flights.  This  is  the  first  objective  of  planners.  This  certain  number  of  flights  called  a 
scheme.  In  addition  to  concern  about  general  pilot  scheme  status,  the  director  has  to  pay 
attention  to  the  type  of  pilots  and  their  status  of  scheme  in  their  related  groups.  The 
second  objective  is  currency  which  is  also  a  constraint  in  a  flight  schedule.  Currency  is 
the  most  important  objective  in  squadrons  because  the  Air  Force  wants  pilots  to  be  ready 
for  combat.  They  can  only  get  this  readiness  information  from  currencies. 

3.1.7  Constraints 

After  introducing  elements  of  the  schedule  we  have  to  consider  limitations  while 
matching  mission-pilot-plane-time.  First,  pilots  can  only  fly  in  two  blocks  of  a  day.  One 
can  be  in  the  morning  and  the  other  one  afternoon  or  one  can  be  afternoon  and  the  other 
one  at  night.  If  somebody  flies  in  the  morning  he/she  can’t  fly  at  night  in  the  same  day. 
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The  reverse  is  also  true  for  this  constraint  so  that  if  somebody  flies  at  night  then  he/she 
can’t  fly  in  the  next  morning  block. 

The  next  constraint  is  about  currencies.  Every  pilot  type  has  their  mission 
currency  program.  For  example,  four-ship  leaders  and  instructors  can  be  current  60  days 
from  mission  1  but  the  others  can  be  45  days.  Considering  these  changing  constraints, 
planners  choose  the  mission  type  and  pilots  to  schedule.  If  a  pilot  is  not  current  for  a 
specific  mission  he/she  can’t  fly  this  mission  in  an  F-16  C.  He/she  can  fly  with  an  F-16  D 
and  an  instructor  pilot  must  be  in  the  rear  cockpit.  After  this  type  of  flight  he/she 
becomes  current  if  the  sortie  is  successful. 

Pilots  have  currency  status  from  individual  missions  and  they  have  general 
currencies  for  day/night  sorties  as  well.  It  is  the  same  as  individual  mission  currencies  but 
it  affects  all  mission  capabilities  for  pilots.  For  instance,  if  a  pilot  doesn’t  fly  a  day/night 
sortie  and  doesn’t  land  the  plane  by  him/herself  in  that  sortie,  he/she  can’t  fly  any  other 
day/night  sortie  with  an  F-16C  so  that  this  pilot  has  to  be  current  for  landing  before 
getting  permission  to  have  day/night  F-16C  missions. 

There  are  limitations  for  maximum  flight  time  and  sortie  counts  for  pilots  to  be 
scheduled  in  a  day/month/year.  For  example  the  rule  may  say  a  pilot  can  fly  no  more  than 
4  hours  and  3  sorties  in  day.  The  planner  must  obey  this  kind  of  maximum  limitations 
and  also  crew  rest  for  pilots.  Crew  rest  is  a  rule  about  minimum  rest  time  for  pilots. 
According  to  the  flight  schedule  nobody  can  be  in  the  squadron  when  he/she  is  supposed 
to  be  resting. 

Duties  during  flights  are  major  constraints  because  the  pilot  on  duty  can’t  be 
scheduled  for  a  flight  mission..  There  are  three  duties  to  be  planned  for  flight  time.  Two 
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of  them  have  to  be  held  by  four-ship  leaders  or  higher  experienced  pilots.  The  other  one 
is  allowed  to  be  held  by  all  pilots  that  have  general  currency. 

3.2  VFT  Implementation 

In  this  section,  the  method  of  VFT  is  implemented  to  solve  the  first  and  second 
step  of  modeling  the  scheduler  problem.  By  using  a  VFT  model,  manually  built 
alternative  schedules  can  be  evaluated  and  other  good  alternatives  can  be  created  by 
means  of  the  structured  values. 

3.2.1  Step  1:  Problem  Identification 

In  this  step  of  the  VFT  method,  the  major  concern  is  identifying  the  right  problem 
and  identifying  it  correctly.  If  we  cannot  name  it  correctly,  it  will  be  a  waste  of  time  to 
solve  the  wrong  problem.  Previous  sections  mention  some  obstacles  that  trouble 
schedulers.  For  example,  having  more  than  one  DM  makes  the  problem  seem  bigger  than 
it  is  and  undefined  objectives  add  complexity  to  the  schedulers’  decision  environment. 
Because  of  these  difficulties,  people  responsible  for  the  schedule  have  to  think  about 
building  a  structure  for  all  their  objectives  and  values  to  get  rid  of  this  complex 
environment.  Nevertheless,  they  need  a  starting  point  to  begin  their  work  for  the 
structure.  They  ought  to  decide  the  top  box  question  first.  Before  deciding  this  vital 
question,  the  known  and  unknown  inputs  must  be  analyzed. 

Previous  parts  stated  some  variables,  constraints  and  limitations  for  the  problem. 
For  instance  pilots,  aircraft,  missions  and  time  periods  are  the  variables.  Before  the 
scheduler  starts  his  or  her  work,  pilot  presences,  currency  data,  aircraft  availabilities, 
approved  mission  types  and  approved  time  periods  are  known.  However,  the  major 
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unknown  part  for  the  planner  is  how  to  match  the  resources  in  order  to  satisfy  all  the 
objectives  and  obey  the  limitations  and  constraints  at  the  same  time.  The  process  can  be 
thought  of  as  two  phases.  First,  the  matches  are  made  and  then  the  matches  are  fitted  to 
the  constraints  and  limitations.  The  first  phase  is  a  decision  analysis  part  and  the  second 
is  a  math  programming  problem.  If  the  matches  are  made  by  a  decision  analysis  model 
which  is  structured  by  the  help  of  decision  makers,  the  major  obstacles  would  exceed  the 
hardships  of  the  problem  and  yet  a  math  model  can  solve  the  other  fraction  of  the 
problem  without  meeting  barriers  of  DMs. 

After  revealing  the  known  and  unknown  inputs  to  scheduler’s  work,  the  top  box 
question  can  be  decided.  It  could  be  “What  is  the  value  of  pilot-mission  match  in  a 
specific  block  of  time?”.  Aircraft  also  might  be  added  to  this  match  but  as  a  resource,  we 
do  not  have  much  differences  and  objectives  to  match  aircraft  to  pilots  or  missions. 
Aircraft  counts  can  be  used  as  a  constraint  or  a  limitation  in  the  same  specific  time 
period.  Finally,  the  problem  identified  for  this  research  is  deciding  the  value  or  score  of 
the  pilot-mission  match  at  the  defined  time  period. 


3.2.2  Step  2:  Create  Value  Hierarchy 

The  next  step  is  asking  the  DMs  for  their  objectives  and  values  and  then 
combining  them  with  the  written  objectives  and  values  of  the  Air  Force.  After  all,  the 
structure  of  the  decision  model  will  be  built  hierarchically  by  means  of  these 
combinations.  While  gathering  values  together,  it  will  be  crucial  to  pay  attention  to  the 
necessary  properties  of  a  value  hierarchy.  As  a  reminder.  Chapter  2  mentions  the 
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desirable  properties  as  completeness,  non-redundancy,  independence,  operability,  and  small 
size.  For  example,  when  caring  about  completeness,  operability  should  be  considered  as  well. 

In  fact.  Step  1  is  the  starting  point  of  Step  2.  After  writing  the  identified  problem  in 
the  top  box  of  the  hierarchy,  the  solicitation  part  begins.  DMs  and  SMEs  are  asked  for  the 
values  and  a  list  of  values  is  considered  to  be  the  first  tier  of  the  value  hierarchy.  Due  to  the 
non-redundancy  property  some  of  the  values  were  ehminated.  Finally,  four  major  values  are 
used  for  composing  the  first  tier. 


Figure  5:  Top  Box  and  First  Tier  of  Vaiue  Hierarchy 

As  shown  in  Figure  5,  Tier  1  consists  of  Readiness,  Frequency,  Improvement  and 
Motivation  values.  These  major  values  are  not  written  exactly  like  this  on  the  formal 
papers  but  while  comparing  Air  Force  objectives  and  values  concerning  schedules, 
similar  goals  take  over. 

The  Air  Force  is  a  weapon  for  countries  while  they  are  defending  themselves 
from  their  enemies.  Because  countries  can  encounter  conflicts  or  crisis  all  of  a  sudden, 
this  vital  weapon  must  be  ready  for  the  fight  all  the  time.  Squadrons  are  the  warrior  unit 
of  the  Air  Force  and  fresh  warriors  mean  a  lot  for  the  game  plans.  So  “Readiness”  is  an 
organizational  goal  and  a  necessity  for  the  squadrons.  If  fighter  squadron  functions  are 
examined  individually,  they  separate  into  two  main  groups.  The  first  one  is  air  to  air  and 
the  other  one  is  air  to  ground.  In  spite  of  the  fact  that  they  are  tasked  differently,  they  still 
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need  to  be  ready  for  other  missions  than  their  specialty.  Hence  the  total  mission  count  for 
pilots  to  fly  regularly  becomes  extremely  large. 

The  Readiness  value  in  this  hierarchy  introduces  the  score  of  the  pilot-mission 
match  by  the  means  of  readiness  in  accordance  with  a  time  period.  The  score  from  this 
branch  tells  how  much  a  specific  pilot  needs  a  specific  mission  for  his  or  her  freshness  in 
a  stated  time  block.  Every  squadron  has  approximately  30  pilots  and  50  mission  types,  so 
the  match  count  will  be  roughly  1500.  After  evaluating  the  readiness  value,  the  scheduler 
will  know  the  effect  of  the  1500  pilot-mission  matches  on  the  squadrons’  readiness. 

Pilots  must  fly  not  only  for  readiness  but  also  for  becoming  a  frequent  flyer. 
There  are  a  certain  number  of  flights  for  pilots  to  fly  in  specific  time  periods.  Schedulers 
pay  attention  to  choose  the  right  missions  for  the  pilots,  but  they  also  want  them  to  fill 
their  scheme  stated  by  the  Air  Force.  There  must  be  a  balance  between  pilots  because  of 
the  mandatory  flight  counts  that  are  revealed  at  the  beginning  of  every  flight  year. 
Besides  this  necessary  balance,  planners  also  have  to  balance  between  the  readiness  of 
pilots  and  the  frequency  of  flying.  For  example,  a  scheduler  sometimes  has  to  match  a 
specific  pilot  to  a  mission  which  he  or  she  flew  one  block  ago  to  increase  this  pilot’s 
flight  count  in  this  month  to  catch  up  with  the  other  pilots’  flight  numbers.  We  can  draw 
another  picture  worse  than  that.  In  the  same  example,  another  pilot  may  need  this  mission 
on  this  specific  block  because  of  currency  problems;  nevertheless,  the  scheduler 
sometimes  chooses  frequency  prior  to  readiness  and  this  pilot  becomes  non-current  due  to 
priorities.  To  sum  up,  the  decision  model  of  the  pilot-mission  match  separates  readiness 
and  frequency  values  as  two  branches  and  needs  a  priority  plan  to  select  one  of  them  if 
the  scheduler  has  to. 
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Another  property  of  the  value  structure  is  completeness,  so  all  concerns  of  the 
scheduler  and  commanders  must  be  evaluated  in  this  structure.  At  the  beginning  of 
Chapter  3,  some  differences  between  pilots  were  revealed.  They  are  separated  into  groups 
by  their  experiences  such  as  instructors,  4-ship  leaders,  2-ship  leaders  and  wingmen.  A 
graduated  pilot  becomes  a  wingman  first  and  after  that  position,  he/she  follows  some 
program  to  be  a  leader.  With  these  programs,  pilots  start  to  improve  and  get  experience  in 
the  squadron.  Another  type  of  improvement  is  getting  used  to  bad  weather  and  night 
conditions  through  categorical  programs.  Furthermore,  pilots  who  have  not  flown  for  a 
long  time  or  newcomers  are  also  subject  to  some  refreshment  and  training  programs  to 
improve  from  their  low  level  position  to  another  pilots’  higher  level  position.  It  is 
important  to  keep  every  individual  pilot  on  track  with  respect  to  any  program  he/she  may 
be  in.  So  the  decision  about  the  pilot-mission  match  is  affected  by  this  improvement 
concern. 

An  important  task  of  commanders  is  being  aware  of  the  psychological  status  of  the 
elements  in  the  squadron.  Commanders  have  to  keep  track  of  pilots’  level  of  motivation 
and  react  early  when  something  is  wrong  with  them  or  their  family.  A  branch  of  pilots’ 
interests  about  the  schedule  is  also  in  the  value  structure  to  track  their  needs  and  hence 
motivation  of  the  squadron  could  be  boosted  by  this  action.  Due  to  qualitative  conditions 
and  unknowns,  this  is  the  hardest  part  for  the  model  to  build,  but  it  is  known  that  a  value 
structure  does  not  evaluate  all  values  equally.  It  uses  weights  solicited  from  DMs  to 
calculate  an  overall  score;  so  defining  a  motivation  branch  can  have  as  much  affect  on 
this  score  as  the  DM’s  want  and  is  a  good  start  for  evaluating  the  motivation  affect  on  the 
pilot-mission  match  decision. 
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3.2.3  Step  3:  Develop  Evaluation  Measures 


This  step  accommodates  two  phases.  Namely,  choosing  the  right  evaluation 
measures  for  the  associated  value  and  deciding  the  range  of  the  evaluation  measure  in 
order  to  use  this  range  in  further  steps.  After  defining  values,  the  work  for  collecting 
measures  to  evaluate  them  starts.  The  following  sections  explain  the  measures  for  the 
four  major  values. 

3.2.3.1  Measures  for  Readiness 

All  commanders  agree  that  the  most  important  value  is  being  ready  for  the  game 
plan.  But  there  are  not  enough  written  rules  to  measure  schedules  in  terms  of  this  value. 
The  only  written  rule  is  currency  limitations  for  missions.  Currency  is  discussed  in 
previous  parts  of  this  section,  and  it  basically  means  a  limitation  of  days  that  a  pilot  can 
fly  a  specific  mission.  A  fighter  squadron  has  roughly  50  missions.  Table-1  shows  an 
example  of  a  fighter  squadron  spreadsheet  of  official  flight  types  and  mission  numbers. 
AA,  AG  and  NI  mean  air  to  air,  air  to  ground  and  night,  respectively.  The  type  column 
groups  the  same  types  of  missions  together;  thus,  there  are  two  numbers  for  all  missions. 
The  first  one  is  the  mission  number  and  the  other  one  is  the  order  number  of  the  specific 
mission  in  its  type. 
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MISSIONS 

TYPES 

MISSIONS 

TYPES 

M-1 

AA-1 

M-26 

AA-9 

M-2 

M-27 

AA-10 

M-3 

M-2S 

AA-11 

M-4 

M-29 

AG-1 

M-5 

M-30 

M-6 

M-31 

M-7 

M-32 

AG- 2 

M-8 

M-33 

M-9 

M-34 

M-10 

AA-2 

M-35 

M-11 

M-36 

M-12 

M-37 

M-13 

M-3S 

AG- 3 

M-14 

M-39 

M-15 

AA-3 

M-40 

AG-4 

M-16 

M-41 

AG- 5 

M-17 

M-42 

AG- 6 

M-IS 

M-43 

M-19 

AA-4 

M-44 

AG- 7 

M-20 

M-45 

Nl 

M-21 

M-46 

M-22 

AA-5 

M-47 

M-23 

AA-6 

M-4S 

M-24 

AA-7 

M-49 

M-25 

AA-S 

M-50 

Table  1 :  An  Example  of  Mission  Numbers  and  Types  Table 

As  far  as  currency  concerns,  official  papers  do  not  define  the  number  of  current 
days  for  every  individual  mission.  First,  there  are  written  currency  limits  for  day  and 
night  missions  generally.  So  nobody  can  fly  a  day/night  mission  if  he/she  is  not  current  in 
terms  of  the  currency  limits  and  this  is  called  a  general  currency  limit  for  the  flights. 
There  are  some  critical  missions  that  have  special  currency  limits.  For  instance,  an  air 
refueling  mission  has  a  special  currency  limit  and  schedulers  keep  track  of  the  last  flight 
date  of  this  mission  for  every  pilot.  Furthermore,  types  of  missions  can  have  a  currency 
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defined  for  them  as  well.  From  Table-1,  AA-1  has  nine  missions  and  standard  operation 
procedures  can  define  a  currency  limit  for  AA-1  itself.  Thus,  when  somebody  flies  one  of 
the  AA-1  missions,  he/she  can  be  current  for  all  of  AA-1  missions.  Finally,  the  Air  Force 
can  state  currencies  according  to  the  experience  level  of  pilots.  While  an  instructor  can  be 
current  for  60  days  for  a  night  mission,  a  wingman  has  to  fly  the  same  mission  every  30 
days.  Because  of  the  complexity  in  currency  calculations,  a  spreadsheet  for  every  pilot  is 
prepared  to  track  the  last  flight  date  of  every  mission. 

The  Air  Force  gives  much  weight  to  currency  limits,  and  the  scheduler  has  to 
watch  these  numbers  carefully.  Because  of  this,  the  readiness  value  contains  currency  as 
an  important  evaluation  measure.  But  currency  does  not  provide  all  of  the  readiness 
anxiety.  Currency  limits  are  somewhat  large  limits  and  if  other  numbers  are  not  defined 
beside  the  currency  limits,  the  scheduler  has  to  choose  which  mission  to  fly  more  than  the 
others.  The  scheduler  not  only  keeps  mission  currencies  fresh  but  also  tries  not  to  keep 
aircraft  on  the  ground.  This  problem  can  be  solved  by  defining  some  other  readiness 
limits  for  specific  missions  that  are  more  important  than  the  others  according  to  the 
fighter  squadron’  game  plan  and  commanders’  view.  The  following  figure  shows  the 
readiness  value  and  its  measures.  DMs  are  asked  for  the  wanted  readiness  limits  pilot  by 
pilot  and  mission  by  mission.  Then  a  spreadsheet  that  tracks  pilots’  wanted  readiness 
level  is  constructed. 
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Readiness 


Figure  6:  Evaluation  Measures  for  Readiness 

3.23.2  Measures  for  Frequency 

Frequency  of  flight  for  pilots  means  how  many  flights  a  specific  pilot  flew  in  a 
specific  time  period.  From  this  starting  point,  we  can  say  that  the  Air  Force  declares 
yearly  and  monthly  schemes  for  pilots  every  year.  In  the  readiness  part  there  is  the 
written  and  strict  measure  of  currency,  but  besides  this  measure,  the  DM  uses  another 
measure  which  is  not  written  and  strict  but  is  needed  for  scheduling.  Frequency  is 
considered  similar  to  readiness  and  so  will  have  official  measures  that  are  yearly  and 
monthly  schemes  declared  by  the  Air  Force  but  schedulers  need  another  measure  for 
weekly  and  daily  concerns. 

Yearly  and  monthly  schemes  can  change  every  year  but  they  are  simply  numbers, 
approximately  120  and  8,  respectively.  This  means  a  pilot  in  the  squadron  has  to  fly  120 
flights  in  this  year,  but  there  is  no  constraint  about  mission  types.  So  a  pilot  can  fly  one 
certain  mission  type  120  times,  and  it  will  not  be  a  problem  from  the  aspect  of  the  Air 
Force  yearly  scheme  considerations. 
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Commanders  have  some  optimum  numbers  of  weekly  flights  and  daily  flights  for 
every  experience  level  in  their  minds.  For  example,  they  can  say  a  wingman  has  to  fly 
everyday  or  define  one  flight  as  the  optimum  number  of  flights  for  everybody.  Thus  zero 
flights  or  two  flights  will  be  penalized  due  to  the  optimum  number.  This  can  also  be  done 
for  weekly  flights.  DMs  are  asked  for  the  optimum  numbers  of  daily  and  weekly  flights; 
then  they  are  applied  to  the  model  as  weekly  wanted  and  daily  wanted  measures  of 
frequency. 
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Figure  7:  Evaluation  Measures  for  Frequency 

Figure  7  shows  the  evaluation  measures  for  the  frequency  value.  The  first  two 
measures  are  taken  from  formal  papers  and  the  other  two  are  asked  from  DMs. 
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3.2.3.3  Measures  for  Improvement 


The  Improvement  value  is  concerned  with  the  effect  a  specific  pilot-mission 


match  has  on  specific  pilot’s  improvement  in  his/her  flight  experience.  DMs  are  asked  for 
how  to  evaluate  improvement  of  pilots,  and  according  to  their  perspective,  the  following 
figure  shows  evaluation  measures  for  the  improvement  value  in  the  pilot-mission  match 
value  hierarchy. 


Improvement 


Figure  8:  Evaluation  Measures  for  Improvement 


The  Career  part  measures  to  what  extent  a  certain  match  helps  pilots’  experience, 
refreshment  or  categorical  programs.  For  example,  if  a  pilot  needs  M-1  for  his/her 
refreshment  program,  this  pilot’s  match  to  M-1  will  be  more  valuable  in  terms  of 
improvement.  According  to  the  SME’s  views,  spreadsheets  are  prepared  for  tracking 
pilots’  levels  in  these  programs  and  every  possible  mission  is  graded  by  this  data. 
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MISSIONS 

CAT-III 

CAT-II 

CAT-1 

BEGINNER 

2  SHIP  LEADER 

4  SHIP  LEADER 

INSTRUCTOR 

REFRESHMENT 

TOTAL 

M-1 

0 

0 

0 

0 

0 

0 

0 

0 

0 

M-13 

1 

0 

0 

1 

0 

0 

0 

0 

1 

M-23 

0 

0 

0 

1 

0 

0 

0 

0 

1 

M-42 

0 

1 

0^ 

0 

1 

0 

0 

0 

1 

M-50 

0' 

0 

O' 

0 

0 

0 

1 

0 

1 

Table  2:  An  Example  of  Career  Spreadsheet  for  Pilot  X 

Table  2  is  an  example  of  the  career  measure  spreadsheet  for  generic  pilots.  From 
this  table,  M-1  has  no  affect  on  the  first  pilot’s  career.  But  the  second  line  shows  a 
newcomer  and  M-13  for  this  newcomer  means  a  lot  in  his  improvement.  The  third  line 
introduces  another  newcomer’s  position  in  the  track;  however,  M-23  only  affects  the 
beginning  program.  The  fifth  line  reveals  a  need  for  M-50  for  a  pilot  who  is  preparing  to 
be  an  instructor.  An  important  property  of  these  programs  is  nobody  can  be  on  more  than 
two  career  programs.  So  the  career  range  is  from  zero  to  two. 

The  second  measure  for  the  improvement  value  is  personal  need.  Every  pilot  has 
debriefing  notebooks.  After  all  flights,  leaders  or  instructors  write  strength  and  weakness 
points  based  on  the  previous  flight.  Using  this  notebook,  the  training  section  can  track 
pilots’  conditions  for  specific  mission  types.  However,  nothing  happens  automatically 
after  a  pilot  does  bad  or  good  in  his/her  previous  flight.  A  range  can  be  defined  for 
evaluating  the  flights  by  leaders  or  instructors  and  then  if  a  pilot  satisfies  his/her  leaders’ 
goals  in  a  specific  mission,  he/she  will  get  a  good  grade.  By  means  of  this  evaluation, 
personal  needs  can  be  revealed  and  future  schedules  built  being  aware  of  the  previous 
flights’  success  or  failure.  A  spreadsheet  is  prepared  for  keeping  the  grades  of  all 
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missions  by  the  training  section,  so  this  model  can  use  this  data  to  evaluate  the  personal 
need  part  of  the  improvement  value. 


MISSIONS 

LAST  FLIGHT  DATE 

PERSONAL 

M-1 

12/13/2010 

5 

M-2 

12/14/2010 

3 

M-3 

12/15/2010 

1 

M-4 

12/16/2010 

2 

M-5 

12/17/2010 

4 

Table  3:  An  Example  Personal  Need  Evaluation  Table 


As  seen  in  Table  3,  a  spreadsheet  for  the  personal  need  part  has  mission  numbers, 
last  flight  date  for  respective  mission  and  evaluation  of  the  mission  by  an  experienced 
pilot.  For  example,  this  pilot  has  no  need  for  M-1  and  M-5,  but  he/she  needs  to  fly  M-3 
again  as  soon  as  possible.  The  range  for  personal  need  measure  is  defined  as  [1-5]. 

3.2.3.4  Measures  for  Motivation 

The  following  figure  introduces  evaluation  measures  for  motivation  value  in  the  model. 


Figure  9:  Evaluation  Measures  for  Motivation 
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Motivation  is  the  hardest  value  to  evaluate  for  pilot-mission  matches.  Every 
commander  or  leader  accepts  that  the  task  for  keeping  pilots  motivated  is  their 
responsibility,  but  they  do  not  have  any  structured  way  for  thinking  of  it.  In  order  to  make 
a  good  start  for  evaluating  the  effect  of  matches  on  motivation,  DMs  and  pilots  are 
solicited  for  the  purpose  of  creating  the  motivation  part  of  the  hierarchy.  Some  of  the 
concerns  that  come  from  pilots  are  the  same.  Thus  the  motivation  part  is  constructed  as 
shown  in  Figure  9. 

Pilots  have  challenging  and  difficult  work  every  day.  Flight  and  office  hours  are 
changing  a  lot  because  of  the  dynamic  environment.  Hence,  they  do  not  have  an  orderly 
family  or  private  life.  Eventually,  they  have  hardships  when  they  need  time  to  take  care 
of  issues  related  to  their  private  life.  At  that  point,  permission  decisions  come  into 
question  and  have  an  effect  on  the  pilot-mission-time  match  evaluation.  Sometimes  pilots 
want  permission  not  to  fly  on  a  specific  day  in  order  to  handle  their  other  activities. 
Nevertheless,  there  currently  is  no  decision  model  for  responding  to  their  desires. 
Commanders  deal  with  these  kind  of  requests  throughout  the  flight  year  and  they  use 
their  conscience  and  logic  together  to  be  fair.  DMs  are  asked  for  their  thoughts  about  how 
they  answer  these  questions.  A  spreadsheet  and  a  formula  are  made  by  using  their  logic 
for  evaluating  the  permission  part  of  the  motivation  value. 


WANT  TO  FLY? 

PAST  REQUESTS 

PAST  RESULTS 

PAST  PERCENTAGE 

0 

1 

1 

1 

PERMISSION 

1 

Table  4:  An  Example  of  a  Spreadsheet  for  Evaluating  Permission  Requests 
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The  above  table  presents  an  example  for  an  evaluation  of  a  permission  request 
made  by  a  generic  pilot  at  a  certain  block  of  time.  The  first  cell  asks  the  pilot  whether 
he/she  wants  to  fly  at  that  time  period  and  answers  are  numbered  as  negative  (0)  and 
positive  (1).  Starting  from  the  second  cell,  data  from  the  past  is  evaluated  and  then  the 
permission  evaluation  result  is  given  in  the  last  cell.  In  this  specific  example,  the  pilot 
does  not  want  to  fly  in  this  block  and  the  permission  calculation  comes  up  with  a  positive 
result  due  to  his/her  past  requests  and  the  negative/positive  replies  to  them.  Finally, 
permission  points  will  affect  the  overall  grade  of  motivation  value  gradually.  Using  a 
structured  way  for  permission  decisions  and  welcoming  pilots  to  participate  in  decisions 
will  help  increase  the  squadrons’  motivation. 

Pilots  from  every  level  of  experience  desire  to  fly  an  equal  number  of  flights  in  a 
month  or  year.  Because  of  this,  they  watch  the  flight  counts  of  the  other  pilots  at  the  same 
level  of  experience.  If  there  is  a  significant  difference  between  them,  they  become  upset. 
In  order  to  evaluate  this  stress  on  pilot -mission  matches,  a  spreadsheet  is  prepared  and 
schedulers  pay  attention  not  to  make  mistakes  about  equality.  When  pilots  are  solicited 
about  the  feelings  from  these  situations,  they  mention  two  types  of  equality  that  are 
important  to  them.  The  first  one  is  yearly  and  the  other  one  is  monthly  equality.  They  are 
asked  about  the  value  of  the  differences  and  the  range  for  monthly  or  yearly  equality  and 
evaluation  measures  are  built.  They  state  that  a  difference  of  two  flights  in  a  month  or 
five  in  a  year  is  not  important  but  above  these  numbers  they  start  to  get  concerned. 


YEARLY  FLIGHT 

MONTHLY  FLIGHT 

YEARLY  MAX 

MONTHLY  MAX 

YEARLY  DIFF 

MONTHLY  DIFF 

19 

3 

35 

5 

16 

2 

Table  5:  An  Example  Spreadsheet  for  Evaluating  Yearly/Monthly  Equality 
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Table  5  shows  an  example  of  evaluating  yearly  and  monthly  equalities.  The  first 
two  cells  are  the  flight  counts  of  a  generic  pilot  in  this  year/month.  The  third  and  fourth 
cells  contain  the  number  of  flights  of  the  pilot  who  flew  more  than  all  others  at  his/her 
level  of  experience.  The  last  two  cells  calculate  the  differences  the  model  will  consider. 
Hence,  the  matches  will  be  graded  according  to  these  equality  concerns.  For  example, 
this  pilot’s  equality  part  will  be  graded  separately  due  to  the  importance  of  yearly  and 
monthly  differences.  The  yearly  difference  is  sixteen  and  it  is  valuable  to  schedule  this 
pilot  to  a  mission  in  this  block,  but  the  monthly  difference  is  two  meaning  there  is  no 
significant  difference  in  this  month  between  pilots  at  this  level  of  experience.  Finally,  this 
pilot  will  get  a  medium  grade  from  a  flight  in  that  block  due  to  the  aim  of  decreasing 
yearly  differences.  For  this  measure,  maximum  numbers  were  chosen  for  the  evaluation 
input  differences  rather  than  averages  as  they  do  not  tell  the  same  story. 


YEARLY  FLIGHT 

MONTHLY  FLIGHT 

YEARLY  MAX 

MONTHLY  MAX 

YEARLY  DIFF 

MONTHLY  DIFF 

19 

3 

35 

5 

16 

1 

YEARLY  AVG 

MONTHLY  AVG 

19 

3 

Table  6:  An  Example  for  Evaluating  by  Average  Numbers 


For  instance,  yearly  and  monthly  average  numbers  are  added  on  the  same 

example  as  shown  in  Table  6.  It  can  be  easily  seen  that  the  same  pilot  would  not  have  a 

good  score  from  a  mission  match  because  his/her  flight  number  is  equal  to  the  average  in 

the  squadron.  Nevertheless,  the  maximum  flight  number  is  sixteen  more  and  the  model 

does  not  evaluate  this  nuance  if  the  average  is  used  as  the  difference  indicator. 

The  final  evaluation  measure  for  motivation  is  duty  equality.  In  fighter  squadrons, 

there  are  several  duties  during  flight  hours  and  related  to  flights  but  only  two  of  them  are 

being  considered  as  important  with  respect  to  the  pilots’  views.  The  Director  of  duties 
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tries  to  pass  out  these  duties  equally  and  fairly.  The  DM  is  asked  for  his  opinion  on 
deciding  fairly  and  according  to  his  views  a  spreadsheet  is  prepared  for  keeping  track  of 
past  duty  data  for  the  purpose  of  evaluating  equality  on  duties. 


DUTYl 

DUTY  2 

TOTAL 

TOTAL  MAX 

DIFFERENCE 

4 

0 

4 

S 

4 

Table  7:  An  Example  Spreadsheet  for  Evaluating  Duty  Equalities 


Because  the  duties  are  during  flight  hours,  a  pilot  who  is  on  duty  cannot  fly.  If  the 
count  of  duties  is  not  equally  distributed,  some  of  the  pilots  will  be  affected  by  the  means 
of  currency  and  frequency.  So,  duty  equality  means  a  lot  for  the  motivation  value.  In 
Table  7,  an  example  spreadsheet  for  evaluating  duty  equalities  is  presented.  This  sample 
pilot  has  had  duty  1  four  times  in  this  flight  year.  The  first  two  cells  indicate  the  counts 
for  duty  1  and  2.  The  third  cell  gives  the  total  number  of  duties  held  by  the  sample  pilot 
and  the  fourth  cell  reveals  the  maximum  total  count  of  duties  held  by  a  pilot  at  the  same 
level  of  experience.  The  fifth  cell  states  the  difference  between  the  sample  pilot  and  the 
pilot  with  the  maximum  number  of  duties.  Using  the  difference,  the  model  will  come  up 
with  an  inverse  ratio  scoring  for  the  duty  equality  part  of  the  motivation  value.  For 
example,  in  Table  7  there  is  a  four  in  the  difference  cell  and  the  sample  pilot  will  have  a 
low  score  from  the  duty  equalities  part  which  means  he/she  can  get  another  duty  because 
he/she  has  less  duty  counts  than  someone  at  his/her  level  in  the  squadron. 

The  following  table  introduces  all  evaluation  measures  and  their  units.  As  an 
example,  currency  is  an  evaluation  measure  with  a  range  [0,120]  and  this  number  shows 
the  remaining  current  days  for  a  pilot.  The  developing  evaluation  measures  step  contains 
ranges  and  units  for  measures  and  is  used  in  further  steps. 
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Range 

Unit 

Currency 

0-120 

Remaining  Current  Days 

Wanted  Readiness 

0-120 

Remaining  Ready  Days 

Yearly  Scheme 

0-1 

Yearly  Ratio  {Need/Presence) 

Monthly  Scheme 

0-1 

Monthly  Ratio  {Need/Presence) 

Weekly  Wanted 

0-1 

Weekly  Ratio  {Need/Presence} 

Daily  Wanted 

0-1 

Daily  Ratio  {Need/Presence) 

Career 

0-2 

Matched  Flight  Count 

Personal  Need 

1-5 

Success  Point  in  Last  Flight 

Permission 

0-1 

Past  Result  Percentage 

Yearly  Equality 

0-30 

Yearly  Flight  Differences  at  Same  Level 

Monthly  Equality 

0-8 

Monthly  Flight  Differences  at  Same  Level 

Duty  Equality 

0-10 

Duty  Differences  at  Same  Level 

Table  8:  Evaluation  Measures  Range  and  Unit  Table 


3.2.4  Step  4:  Create  Value  Functions 

The  previous  step  collects  all  evaluation  measures  for  the  value  hierarchy  and 
decides  on  the  ranges  and  units  of  these  measures.  As  can  be  seen  in  Table  8,  the 
measures  have  different  ranges  and  units.  However,  in  order  to  get  an  overall  score  from 
the  structure,  we  need  to  have  similar  units  and  ranges  for  each  evaluation  measure.  The 
Single  Dimensional  Value  Function  (SDVF)  changes  the  different  ranges  and  units  to 
similar  ones.  It  uses  a  zero  to  one  range  and  gives  a  score  that  can  be  thought  of  as  a  ratio 
of  satisfaction  of  the  values.  There  are  two  classes  of  SDVF.  The  first  one  is  continuous 
and  converts  an  infinite  number  of  inputs  into  the  [0,  1]  range.  The  other  class  uses 
categorical  scales,  and  it  is  called  a  discrete  SDVF.  If  the  number  of  possible  inputs  is 
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small  and  certain,  the  discrete  SDVF  is  more  suitable.  In  this  research,  ten  out  of  the 


twelve  value  functions  are  created  using  exponential  functions  as  continuous  SDVFs. 

l-expHAv 

V,(A.)= 

,olhenvi!ie 

J.  ”  Jf; 

(2) 

Equation  (2)  is  the  equation  for  monotonically  increasing  value  functions. 
Monotonically  increasing  means  the  value  increases  as  the  score  increases  and  can  never 
be  less  than  a  previous  value  (Kirkwood,  1997). 


'[-exp[-ur-A,VA]_^ 

l-exp[-(x, 

^  ,oth  envis  e 

:c.  — 

(3) 

Equation  (3)  is  the  equation  for  monotonically  decreasing  value  functions 
(Kirkwood,  1997). 

Where: 

Vi  (xi)  =  the  exponential  single  dimensional  value  function  for  alternative  x  on  measure  i 

Xi  =  score  for  alternative  x  on  measure  i 

Xi^  =  the  upper  bound  for  alternative  x  on  measure  i 

Xi^  =  the  lower  bound  for  alternative  x  on  measure  i 

pi  =  exponential  constant  for  measure  i 
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The  value  functions  in  this  research  are  constructed  with  the  help  of  DMs  and 
SMEs  from  the  squadrons.  A  computer  software  program  is  used  to  facilitate  the  process. 
This  research  uses  the  Hierarchy  Builder  2.0  software  program  which  helps  build  the 
value  hierarchy  at  every  step  of  value  focused  thinking  (Weir  J. ,  2008). 

3.2.4.1  Value  Functions  for  Evaluation  of  Readiness 


Under  the  readiness  value  there  are  two  evaluation  measures  named  currency  and 
wanted  readiness.  Both  of  them  have  monotonically  decreasing  piecewise  exponential 
value  functions. 


1  i 
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Figure  10:  Value  Function  for  Currency 
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Figure  10  shows  the  value  function  for  currency  which  has  a  range  of  [0,120]  on 
the  X  axis  and  [0,  1]  on  the  Y  axis.  Using  this  function,  the  currency  measure’s  range  is 
converted  to  the  general  range  of  a  value  model.  DMs  are  asked  to  decide  on  several 
critical  numbers  of  days  while  they  are  scheduling  and  thinking  about  currency. 
According  to  the  DMs,  the  critical  points  are  7,  14,  21,  28  and  60  days  before  being  non- 
current.  So,  one  week  before  is  the  most  critical  time  to  schedule  a  pilot  to  a  mission 
because  schedulers  prepare  weekly  schedules.  The  other  critical  days  are  also  for 
reminding  schedulers  that  the  last  days  are  coming  before  becoming  non-current.  Sixty 
days  is  the  point  in  time  that  means  a  specific  pilot  does  not  need  the  mission  until  sixty 
days  has  passed  after  his  last  flight  for  that  certain  mission. 

The  wanted  readiness  part  of  the  same  value  is  converted  with  a  similar  value 
function.  All  value  functions  used  and  the  spreadsheets  with  the  necessary  calculations 
are  presented  in  Appendix  B. 

3.2. 4.2  Value  Functions  for  Evaluation  of  Frequency 

The  frequency  value  is  evaluated  by  four  measures  and  these  measures  have 
similar  value  functions.  As  an  example,  a  yearly  scheme  is  selected  and  introduced  here. 
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Figure  11 :  Value  Function  for  Yearly  Scheme 

The  previous  paragraphs  discussed  the  yearly  scheme  and  how  it  is  calculated. 
After  calculation  of  the  yearly  scheme  and  yearly  ratio  by  the  respective  spreadsheet  of  a 
pilot,  the  continuous  SDVF  is  used  to  get  the  value  of  the  yearly  scheme.  DMs  agreed 
that  all  percentages  of  yearly  ratio  should  be  the  same  as  the  value.  For  instance  a  yearly 
ratio  of  0.5  gives  a  0.5  as  the  yearly  scheme  value.  The  rest  of  the  evaluation  measures’ 
SDVFs  for  frequency  are  formed  similarly.  All  value  functions,  spreadsheets  and 
necessary  calculations  are  presented  in  Appendix  B. 
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3.2.43  Value  Functions  for  Evaluation  of  Improvement 


The  Improvement  value  introduces  two  evaluation  measures.  They  are  career  and 
personal  need.  After  the  DMs  are  solicited,  SDVFs  for  these  measures  showed  up  as 
categorical  scaled  value  functions.  As  an  example,  the  personal  need  part  is  shown  here. 


Figure  12:  Value  Function  for  Personal  Need 

Figure  12  shows  the  SDVF  for  personal  need,  and  it  is  a  discrete  value  function 
containing  five  categories.  As  stated  in  the  previous  paragraphs,  the  personal  need 
spreadsheet  is  created  by  the  training  section  of  squadrons,  and  it  keeps  the  grades  of  the 
last  flights  related  to  the  specific  pilot.  After  evaluation  of  the  flight,  leaders  or  instructors 
enter  the  grades  into  the  spreadsheet  and  the  grades  are  inputs  for  the  personal  need 
evaluation  measure  of  the  improvement  value.  For  example  if  a  pilot  receives  a  four  from 
his/her  last  flight  of  a  certain  mission,  he/she  gets  0.25  as  a  score  for  personal  need.  This 
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means  he  or  she  does  not  need  to  fly  this  certain  mission  for  a  while.  The  Career  part  is 
almost  the  same.  All  value  functions,  spreadsheets  and  necessary  calculations  are 
presented  in  Appendix  B. 

3.2.4.4  Value  Functions  for  Evaluation  of  Motivation 

The  motivation  value  has  four  evaluation  measures;  permission,  yearly  equality, 
monthly  equality  and  duty  equality.  The  first  three  measures’  SDVFs  are  monotonically 
increasing  and  similar  to  each  other  when  calculating  the  values.  Due  to  this  similarity, 
monthly  equality  is  shown  as  an  example  of  the  three  value  functions. 

The  following  figure  is  the  exponential  function  of  monthly  equality.  According 
to  the  DMs’  opinions,  there  are  two  critical  points  in  this.  The  range  for  monthly  equality 
is  [0,  8]  and  the  unit  is  the  difference  of  flight  count  at  the  same  level  of  experience.  DMs 
and  pilots  agreed  that  there  is  no  significant  problem  with  the  differences  between  zero 
and  two,  but  after  that  point,  the  importance  of  the  difference  rises  quickly  until  the  point 
of  difference  in  the  flight  count  is  six. 
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Monthly  Equality 


Monthly  Flight  Dtffe ranees  at  Same  L^vd_ j 

Figure  13:  Value  Function  of  Monthly  Equality 

DMs  and  pilots  are  asked  to  evaluate  the  points  two  and  six  to  get  the  values 
shown  in  Figure  13. 

Another  evaluation  measure  for  motivation  is  duty  equality.  This  measure’s 
SDVF  is  a  decreasing  continuous  function.  The  spreadsheet  for  tracking  the  equality 
among  pilots  keeps  the  total  amount  of  duties  that  every  pilot  held  in  this  flight  year  and 
then  calculates  the  differences  of  duty  counts  between  every  pilot  and  the  pilot  at  same 
experience  level  who  was  on  duty  the  most.  According  to  this  number,  the  scores  for  the 
duty  equality  part  are  acquired. 
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Duty  Equality 


Differgnces  at  Same  Level  j 

Figure  14:  Value  Function  for  Duty  Equality 

The  critical  points  are  determined  with  respect  to  the  pilots’  point  of  view. 
According  to  them,  the  value  decreases  quickly  as  the  differences  decrease  to  five.  Then 
from  five  to  eight,  the  values  of  differences  decrease  less  quickly.  Finally  the  last  point  is 
ten  because  all  of  the  pilots  agreed  that  differences  higher  than  ten  do  not  change 
motivation.  The  ratios  are  solicited  from  pilots  and  the  Y  axis  is  created  with  the  help  of 
their  inputs. 

Two  of  the  four  evaluation  measures  for  motivation  are  shown  in  this  part  as 
examples.  However,  all  value  functions,  spreadsheets  and  necessary  calculations  are 
presented  in  Appendix  B. 
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3.2.5  Step  5:  Weight  Value  Hierarchy 


At  this  step,  building  the  value  hierarchy  is  finished  after  prioritizing  and  ordering 
the  values  by  the  preferences  of  the  DM.  Because  the  values  do  not  have  the  same 
importance,  they  must  be  weighted  in  order  to  sum  the  scores  according  to  their  priorities. 
There  are  two  ways  to  weight  the  value  hierarchy  as  mentioned  in  Chapter  2.  In  this 
research,  local  weighting  is  used  due  to  large  number  of  branches  and  then  the  four 
branches  are  weighted  individually.  With  the  help  of  the  Hierarchy  Builder  2.0  software 
program,  weights  are  decided  in  accordance  with  the  DMs’  view.  This  research  uses  the 
Analytic  Hierarchy  Process  (AHP)  method  in  order  to  obtain  the  weights.  The  following 
figures  introduce  how  the  AHP  method  gives  the  weights  for  the  motivation  value  with 
the  help  of  Hierarchy  Builder  (Weir  J.  ,  2008).  The  AHP  method  has  DMs  give  their 
preferences  between  two  values’  importance  as  a  number  scaled  one  to  nine.  The  number 
scale  is  given  a  meaning  by  the  degree  of  favorability  of  one  value  over  another.  Figure 
15  displays  the  AHP  weightings  of  the  DM  for  this  research  in  terms  of  the  motivation 
Value. 
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Figure  15:  Pairwise  Comparisons  of  the  Motivation  Vaiue  Evaiuation  Measures 


Permission  is  quite  slightly  favored  compared  to  yearly  equality  and  it  is  slightly 
favor  compared  to  monthly  equality.  According  to  pilots,  duty  equality  is  more  important 
than  permission  but  the  importance  difference  is  really  small.  Pilots  are  asked  about  the 
comparison  of  importance  between  monthly  and  yearly  equality  of  flight  counts  and  the 
answers  are  generally  in  direction  of  yearly  equality  but  the  difference  is  not  even 
slightly,  so  it  is  scaled  as  two.  Pilots  admit  that  the  duty  equality  looks  weightier  than 
yearly  equality  but  the  degree  of  priority  is  nearly  the  same.  At  last,  duty  equality  is  said 
to  be  slightly  favor  in  comparison  to  monthly  equality.  After  all  comparisons  finish,  the 
program  gives  the  weights  in  accordance  with  assessments  above. 


Measure  Name 

Local  Weight 

Permission 

0.293 

Yearly  Equality 

0.185 

Monthly  Equality 

0.107 

Duty  Equality 

0.415 

Table  9:  Motivation  Branch  Local  Weights 


Table  9  reveals  the  local  weights  for  the  motivation  value  branch.  Before 
continuing  for  the  other  branches,  DMs  are  solicited  for  their  satisfaction  about  the  local 
weights  and  if  they  agree  with  the  outputs,  so  this  may  be  an  iterative  process.  Finally,  all 
weights  are  gathered  and  converted  to  global  weights.  The  following  table  shows  all 
twelve  measures  and  their  weights.  According  to  their  weights,  currency  is  the  most 
important  and  monthly  equality  is  the  least  important  measure  in  the  value  hierarchy. 
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Measures 

Weights 

Currency 

0.382 

Yearly  Scheme 

0.170 

Wanted  Readiness 

0.127 

Monthly  Scheme 

0.097 

Career 

0.088 

Daily  Wanted 

0.039 

Personal  Need 

0.022 

Duty  Equality 

0.022 

Weekly  Wanted 

0.020 

Permission 

0.015 

Yearly  Equality 

0.010 

Monthly  Equality 

0.006 

Table  10:  Global  Weights  of  Value  Hierarchy 


Weights  calculated  with  the  help  of  DMs’  opinions  are  used  for  evaluating 
alternatives  and  obtaining  the  overall  value  of  the  alternatives. 


3.2.6  Step  6:  Alternative  Generation 

The  alternative  generation  step  introduces  the  task  of  collecting  all  possible  pilot- 
mission  matches  in  this  research.  Because  fighter  squadrons  have  approximately  30  pilots 
and  50  missions,  the  count  for  possible  matches  will  be  1500.  The  following  table  shows 
some  examples  of  generated  pilot-mission  matches. 
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Alternative  Name 

P1-M45 

P10-M46 

P21-M47 

P32-M48 

P5-M49 

P50-M26 

P43-M50 

P2-M45 

P2-M46 

P2-M47 

P2-M48 

Table  11 :  An  Example  Table  of  Possible  Matches 
3.2.6  Step  7:  Alternative  Scoring 

The  only  step  remaining  before  analyzing  alternatives  is  finding  data  related  to  the 
alternatives.  A  spreadsheet  is  used  to  automatically  transfer  the  data  from  all  pilots’  track 
files.  The  following  table  has  sample  matches  and  needed  inputs  for  the  alternatives. 


Alternative  Name 

Currency 

W.Readiness 

Y.  Scheme 

M.  Scheme 

W.  Wanted 

D.  Wanted 

Pl-Ml 

LW 

100 

0.70 

0.40 

0.3 

0.33 

P2-M4 

33 

90 

0.50 

0.30 

0.6 

0.66 

P15-M3 

87 

87 

0.40 

0.65 

0.7 

0.33 

P10-M4 

5 

5 

0.90 

0.31 

0.3 

0.66 

P22-M5 

56 

45 

0.30 

0.44 

0.1 

0.33 

Table  12:  An  Example  of  Inputs  Spreadsheet  for  Sample  Alternatives  Part-1 
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Alternative  Name 

Career 

P.  Need 

Permission 

Y.  Equality 

M.  Equality 

D.Equaiity 

Pl-Ml 

1 

1 

1 

15 

2 

0 

PZ-M4 

0 

2 

0.55 

1 

4 

10 

P15-M3 

2 

3 

0 

30 

6 

2 

PM)-M4 

0 

4 

0.33 

0 

7 

5 

PZZ-M5 

1 

5 

0.1 

24 

8 

8 

Table  13:  An  Example  of  Inputs  Spreadsheet  for  Sample  Alternatives  Part-2 


3.3  Summary 

In  this  chapter,  the  definitions  and  specifications  of  flight  scheduling  in  fighter 
squadrons  was  stated  and  the  VFT  method  was  implemented  for  modeling  solution  to  this 
problem.  In  the  next  chapter,  the  model  is  verified  and  validated. 
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4.  RESULTS  AND  ANALYSIS 


This  chapter  contains  verification  and  validation  for  the  research  model.  This 
decision  model  for  the  pilot-mission  assignment  is  verified  and  validated  using  the 
deterministic  and  sensitivity  analysis  part  of  VFT  and  the  DM’s  feedbaek. 

4.1  Verification  of  the  Research  Model 

In  this  research,  the  Hierarchy  Builder  2.0  software  program  is  used  for  building 
the  decision  model.  After  the  construction  steps,  in  order  to  calculate  the  scores  of 
evaluation  measures  and  overall  scores,  the  same  program  is  used  but  some 
embellishments  were  made  for  the  purpose  of  accelerating  the  analysis  steps  of  VFT. 
This  verification  part  considers  these  new  additions  made  by  the  author.  However,  one 
sample  evaluation  measure  is  calculated  manually  to  verify  the  software  program’s  model 
first. 

The  sample  evaluation  measure  selected  is  currency.  As  mentioned  in  Chapter  3, 
currency  has  a  monotonically  decreasing  function.  In  this  part  of  the  verification  section, 
some  numbers  of  days  are  taken  as  sample  inputs  and  then  scores  of  the  currency 
measure  are  evaluated  manually  and  by  the  model  to  compare. 

Equation  (3)  is  used  to  calculate  the  value  of  a  point  in  a  monotonically 
decreasing  function.  So,  currency  scores  are  graded  by  this  function.  Nevertheless,  we  do 
not  know  all  the  entries  in  that  formula.  We  need  to  calculate  the  p  value  which  is  the 
exponential  constant.  Equation  (3)  assumes  that  the  highest  entry  gives  zero  and  the 
lowest  entry  gives  one.  However,  most  of  the  SDVE  created  in  this  research  are  made  of 
several  parts  that  do  not  start  with  zero  and  end  with  one.  Due  to  this  fact,  the  formula 
must  be  changed.  First,  to  scale  the  value  function,  the  difference  between  two  extreme 
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points’  scores  is  multiplied  with  it  and  then  the  point  where  the  lowest  score  of  value 


function  occurs  is  added  to  this  function.  Therefore,  the  new  formula  is: 

x^-x 

VM  =  J_,L  X  [VdxX)  -  V(x‘--)]  +  V{x‘--)  (5) 

1-e  P 


With  the  currency  SDVF,  the  value  of  28  days  and  21  and  60  days  is  solicited 
from  the  DM.  The  score  of  28  days  is  0.25  while  21  and  60  were  0.5  and  0.05, 
respectively.  Using  these  values,  the  p  value  of  this  part  of  the  function  is  calculated. 
Then  this  p  value  is  used  to  calculate  values  of  other  days  between  21  and  60.  Using 
Excel  and  Equation  (5),  the  p  value  of  the  scaled  function  was  obtained.  At  first,  Excel 
Goal  Seek  is  used  to  approximate  the  p  value  and  then  around  this  value  a  local  search  is 
made  to  get  the  exact  value  of  28  days.  The  following  table  shows  the  p  values  and  the 
value  of  28  days  using  the  corresponding  p  value. 


RHO 

V(28) 

1 

-8. 7S134S  773940 7200 

0.25000000000001300 

2 

-S.79I34S7739406200 

0.25000000000001200 

3 

-S.79I3487739405200 

0.25000000000001000 

4 

-8. 79 1348  7739404200 

0.25000000000000800 

5 

-8.7913487739403200 

0.25000000000000600 

6 

-8.7913487739402200 

0.25000000000000500 

7 

-8.7913487739401200 

0.25000000000000300 

S 

-8. 7913487739400200 

a 25000000000000100 

S 

-8. 7913487739399200 

0.24999999999999900 

10 

-8.7913487739398200 

0.24999999999999800 

Table  14:  Local  Search  for  Rho  Value 
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As  can  be  seen  from  Table  14,  the  two  p  values  that  are  the  closest  scores  for  28 
days  are  the  8*  or  9'*’  ,  but  we  do  not  need  that  much  accuracy  for  this  research.  After 
obtaining  the  p  value,  the  following  currency  scores  were  calculated  manually  and  by  the 
model. 


Number  of  Days 

Currency  Scores-l 

Currency  Scores-2 

Differences 

21 

0.5 

0.5 

0 

25 

0. 3  3  35  3  24S  68  640  3  600 

0.33362786960279600 

-0.00009537273875954 

30 

0.20820785604108200 

0.20832609176335700 

-0.00011823572227548 

35 

0.13724429267665000 

0.13734408217385400 

-0.00009978949720432 

40 

0.09706203248975020 

0.09713359376545740 

-0.00007156127570715 

45 

0.07430931324655370 

0.07435481661625520 

-0.00004550336970152 

50 

0.06142586093322400 

0.06145090251446420 

-0.00002504158124018 

55 

0.05413076175251970 

0.05414098424906550 

-0.00001022249654578 

60 

0.05 

0.05 

0 

Average  Difference 

-0.00005174740904822 

Table  15:  Currency  Scores  Calculated  By  Manual  versus  By  Model 


The  Currency  Scores- 1  column  gives  the  scores  calculated  manually  and  the 
Currency  Scores-2  column  shows  the  scores  calculated  by  the  software.  The  differences 
are  small  and  the  average  of  the  differences  is  not  significant  for  this  research  problem. 
More  accurate  p  values  could  be  used,  but  it  is  not  necessary  to  use  five  digits  or  more  in 
this  model.  This  model  aims  to  construct  a  big  picture  for  the  DM,  so  using  five  digits  or 
more  would  make  this  picture  more  complicated,  not  less. 

To  continue  the  verification,  ten  alternatives  were  prepared  randomly.  All 
evaluation  measures’  inputs  were  obtained  with  the  help  of  the  Excel  RandBetween 
function.  The  following  tables  show  the  results  measure  by  measure.  They  contain  a 
column  for  inputs,  a  column  for  the  scores  by  this  research’s  model  (i.e.  currency- 1)  and 
another  column  for  the  scores  by  the  software  (i.e.  currency-2). 
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Alternative  Xame 

Currency 

Currency-1 

Currency-2 

Wanted  Readiness 

Wanted  Readiness-1 

Wanted  Readiness-2 

1 

43 

0.081952021 

0.081952021 

24 

0.368157198 

0.368157198 

2 

39 

0.103458454 

0.103458454 

34 

0.148167262 

0.148167262 

5 

47 

0.06830238 

0.06830238 

36 

0.127071134 

0.127071134 

4 

75 

0.037518744 

0.037518744 

54 

0.055249201 

0.055249201 

5 

50 

0.061450903 

0.061450903 

17 

0.630447245 

0.630447245 

6 

68 

0.043344883 

0.043344883 

62 

0.048336017 

0.048336017 

7 

109 

0.009181645 

0.009181645 

44 

0.077773951 

0.077773951 

S 

58 

0.051381106 

0.051381106 

45 

0.074206644 

0.074206644 

9 

35 

0.137344082 

0.137344082 

27 

0.27448839 

0.27448839 

10 

50 

0.061450903 

0.061450903 

38 

0.11027426 

0.11027426 

Table  16:  Verification  of  Readiness  Part 

Table  16  shows  the  evaluation  measure  calculations  of  the  readiness  value  with 
two  methods.  According  to  the  scores,  it  is  easily  seen  that  the  two  methods  give  the 
same  results. 


.Alternative  Xarne 

Yearly  Scheme 

Yearly  Scheme-1 

Yearly  Scheme-2 

Monthly  Scheme 

Monthly  Scheme-1 

Monthly  Scheme-2 

1 

0.24 

0.24 

0.240608351 

0.19 

0.19 

0.190513353 

2 

0.27 

0.27 

0.270657335 

0.69 

0.69 

0.690712698 

3 

0.93 

0.93 

0.930216793 

0.78 

0.78 

0.780571644 

4 

0.83 

0.83 

0.830469988 

0.99 

0.99 

0.990032964 

5 

0.12 

0'.12 

0.120352297 

0.01 

0.01 

0.010033036 

6 

0.3 

0.3 

0.300700311 

0.74 

0.74 

0.740640991 

7 

0.38 

0.38 

0.380785542 

0.46 

0.46 

0.460828073 

S 

0.26 

0.26 

0.260641675 

0.5 

0.5 

0.500833333 

9 

0.64 

0.64 

0.64076776 

0.9 

0.9 

0.900299733 

10 

0.63 

0.63 

0.630776775 

0.68 

0.68 

0.680725043 

Table  17:  Verification  of  Frequency  Part-1 
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.Alternative  >"ame 

Weekly  Wanted 

Weekly  Wanted  ! 

Weekly  Wanted-2 

Daily  Wanted 

Daily  Wanted  ! 

Daily  Wanted-2 

1 

0.45 

0.45 

0.450825091 

0.3 

0.3 

0.300700311 

2 

0.9 

0.9 

0.900299733 

0.42 

0.42 

0.420812144 

3 

0.36 

0.36 

0.360768238 

0.97 

0.97 

0.970096899 

4 

0.51 

0.51 

0.510832981 

0.3 

0.3 

0.300700311 

5 

0.81 

0.81 

0.810512646 

0.2 

0.2 

0.200533689 

6 

0.1 

0.1 

0.100300267 

0.87 

0.87 

0.87037669 

1 

0.47 

0.47 

0.470830388 

0.05 

0.05 

0.050158492 

8 

0.63 

0.63 

0.630776775 

0.7 

0.7 

0.700699688 

9 

0.62 

0.62 

0.620785123 

0.39 

0.39 

0.390793193 

10 

0'.93 

0.93 

0.930216793 

0.82 

0.82 

0.82049165 

Table  18:  Verification  of  Frequency  Part-2 


Table  17  and  Table  18  compare  the  results  for  the  frequency  part  of  the  value 
model.  Considering  the  scores  from  the  two  models,  there  are  slight  differences  on  all 
alternatives.  This  slight  change  is  based  on  the  different  processes  of  determining  the  p 
values.  The  differences  in  these  frequency  calculations  are  shown  in  the  example  below. 


YEARLY  FLIGHT 

YEAR  LY  SC  HEME 

YEARLY  PRESENCE 

19 

120 

173 

YEARLY  RATIO 

0.5S  38 15029 

YEARLY  FLIGHT 

YEARLY  SCHEME 

YEARLY  PRESENCE 

Input 

Input 

Input 

YEARLY  RATIO 

IF{{Y0arly  Scha me- Yearly  Flight) /Yearly 
Presence>OvlF{{Yearly  Scheme-Yearly  Flightj/Yearly 
Presence<14Yearly  Scheme-Yearly  Flightj/Yearly 
Presencejl),0) 

Table  19:  An  Example  of  Yearly  Scheme  Calculation 
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Table  19  is  an  example  of  the  yearly  scheme  calculation  spreadsheet.  The  top  part 
has  the  inputs  from  the  SMEs  and  the  lower  part  calculates  the  ratio  according  to  the 
formula  that  Table  19  shows.  The  difference  between  models  starts  from  this  step  on.  The 
software  uses  the  calculated  yearly  ratio  as  an  input  and  gives  an  output  according  to  the 
SDVF.  The  new  model  uses  the  yearly  ratio  as  the  score  of  yearly  scheme  directly.  To 
get  rid  of  an  unnecessary  step  in  the  process,  the  ratios  are  accepted  as  outputs  of 
frequency  evaluations. 

Furthermore,  Hierarchy  Builder  2.0  uses  an  approximation  method  for  finding  p 
values.  Because  the  SDVFs  of  the  frequency  evaluation  measures  are  linear  functions,  the 
software  program  must  use  infinity  as  its  p  value.  Instead,  it  tries  to  use  a  large  number  in 
order  to  simulate  infinity  so  the  results  for  the  scores  from  the  frequency  are  different 


from  the  real  outputs. 


.Alternative  >"ame 

Career 

Career-1 

Career-2 

Personal  Xeed 

Personal  >"eed-l 

Personal  >"eed-2 

1 

0 

0 

0 

3 

0.5 

0.5 

2 

1 

0.75 

0.75 

1 

1 

1 

3 

0 

0 

0 

4 

0.25 

0.25 

4 

1 

1 

1 

1 

0.75 

0.75 

5 

1 

0.75 

0.75 

1 

1 

1 

6 

0 

0 

0 

1 

1 

1 

1 

0 

0 

0 

3 

0.5 

0.5 

8 

0 

0 

0 

4 

0.25 

0.25 

9 

1 

0.75 

0.75 

1 

0.75 

0.75 

10 

0 

0 

0 

1 

1 

1 

Table  20:  Verification  of  Improvement  Part 


Table  20  verifies  that  there  is  no  difference  between  the  two  evaluation  models  in 
terms  of  the  improvement  part  calculations. 
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.Alternatire  >'ame 

PermissioD 

PermissioD-l 

PermissioD-2 

Yearly  Equality 

Yearly  Equality  ! 

Yearly  Equality -2 

1 

0.S3 

0.83 

0.830469988 

7 

0.310593114 

0.310593114 

2 

0.27 

0.27 

0.270657335 

3 

0.057767028 

0.057767028 

5 

O-.Sl 

0.91 

0.910272751 

17 

0.834528359 

0.834528359 

4 

0.04 

0.04 

0.040128131 

26 

0.973983941 

0.973983941 

5 

0.55 

0.55 

0.550824908 

2 

0.037794401 

0.037794401 

6 

0.42 

0.42 

0.420812144 

11 

0.600722701 

0.600722701 

1 

0.96 

0.96 

0.960127869 

15 

0.775343985 

0.775343985 

8 

0.85 

0.85 

0.850424669 

10 

0.541364125 

0.541364125 

9 

0.4 

0.4 

0.400800177 

0 

0 

0 

10 

0.29 

0.29 

0.290686653 

23 

0.943700732 

0.943700732 

Table  21 :  Verification  of  Motivation  Part-1 


.Alternative  Xame 

Monthly  Equality 

Monthly  Equality  ! 

Monthly  Equality-2 

Duty  Equality 

Duty  Equality  ! 

Duty  Equality-2 

1 

8 

1 

1 

4 

0.682034573 

0.682034573 

2 

8 

1 

1 

7 

0.145621521 

0.145621521 

3 

1 

0.043687038 

0.043687038 

7 

0.145621521 

0.145621521 

4 

6 

0.900118335 

0.900118335 

4 

0.682034573 

0.682034573 

5 

3 

0.441409553 

0.441409553 

8 

0.05 

0.05 

6 

0 

0 

0 

7 

0.145621521 

0.145621521 

7 

6 

0.900118335 

0.900118335 

5 

0.25 

0.25 

8 

6 

0.900118335 

0.900118335 

2 

0.949857834 

0.949857834 

4 

0.662985592 

0.662985592 

8 

0.05 

0.05 

10 

8 

1 

1 

1 

0.984856436 

0.984856436 

Table  22:  Verification  of  Motivation  Part-2 


Table  21  and  Table  22  help  investigate  the  motivation  part  of  the  value  model. 
The  only  difference  is  between  the  Permission- 1  and  Permission-2  columns.  These 
insignificant  changes  depend  on  the  same  reasoning  as  the  frequency  part  calculations’ 
differences  discussed  above.  Hence,  an  unnecessary  step  is  omitted  in  the  process  once 
again,  and  the  new  model  ends  with  approximately  the  same  results  as  the  software 
calculates. 
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Alternative  >”ame 

Overall-1 

Overall-2 

Differences 

1 

0.2<}614732S 

0.206353156 

-0.000205228 

2 

0.30S54S01 

0.308  77854 

-0.00022953 

3 

0.353214507 

0.353330846 

-0.000116339 

4 

0.416441521 

0.416571285 

-0.000129764 

5 

0.2504S04S 

0.250597921 

-0.000107431 

6 

0.21SS79931 

0.220095005 

-0.000215074 

7 

0.178536528 

0.178776266 

-0.000239438 

3 

0.212506489 

0.212746846 

-0.000240357 

P 

0.405898459 

0.406117921 

-0.000219462 

10 

0.325420604 

0.32565  7743 

-0.00023714 

Average  Difference 

-0.000193976 

Table  23:  Verification  of  Overall  Scores 


Table  23  shows  that  there  are  only  small  differences  in  the  fourth  decimal  place  of 
the  scores.  Besides,  the  average  difference  verifies  that  there  is  no  significant  difference 
considering  this  research’s  goals. 


4.2  Validation  of  the  Research  Model 

For  the  purpose  of  validating  the  decision  model  of  the  assignment  problem,  this 
research  uses  four  steps: 

1.  Creating  ten  different  groups  of  five  assignment  alternatives. 

2.  Soliciting  the  DM  to  order  these  alternatives  in  terms  of  his  preferences. 

3.  Using  the  value  model  to  order  these  alternatives  with  the  help  of  the 
deterministic  analysis  part  of  VFT. 

4.  Comparing  the  results 

After  comparing  the  results,  the  model  is  valid  if  there  is  no  difference.  But  if 
there  is  a  difference,  the  validation  process  involves  two  steps.  First,  using  sensitivity 
analysis  for  minor  differences  to  try  and  determine  if  the  weights  are  correct.  While  doing 
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this,  the  DM  may  change  his  weights  after  seeing  the  structured  values  of  the  model.  The 
other  way  is  for  large  differences  to  determine  if  the  model  is  missing  some  evaluation 
criterion.  Hence,  the  model  must  be  changed  with  the  help  of  the  DM’s  feedbacks  and 
then  the  validation  starts  again. 

The  first  step  of  the  validation  process  is  preparing  randomized  alternatives.  In 
this  step,  ten  groups  of  five  alternatives  are  created  with  the  Excel  RandBetween 
function.  Furthermore,  these  groups  were  designed  in  several  difficulty  levels  in  order  to 
begin  solving  simple  assignment  problems  and  then  harder  ones.  For  instance,  there  are 
many  fixed  measures  in  the  simple  problems  so  the  DM  has  to  consider  only  two  or  three 
measures  or  values  to  rank  the  five  alternatives.  The  validation  alternatives  are  also 
constructed  in  two  types.  The  first  type  takes  a  pilot  and  considers  the  available  mission 
matches  for  that  pilot.  However,  the  second  type  chooses  five  different  pilots  and  five 
different  available  missions  to  assign  these  pilots,  respectively. 

4.2.1  Comparing  Mission  Assignments  for  a  Specific  Pilot 


In  this  part,  five  problems  were  created.  The  first  problem  contains  currency  and 
wanted  readiness  data  that  were  randomly  selected.  The  other  data  needed  for  these  five 
assignments  were  chosen  as  fixed  entries. 


Missions 

Current  Days 

Wanted  Days 

Currency  Left: 

Wanted  Left 

M-1 

120 

30 

67 

0 

M-2 

90 

60 

67 

37 

M-3 

60 

60 

44 

44 

M-4 

45 

30 

36 

21 

M-5 

30 

21 

2S 

19 

Table  24:  Problem-1  Inputs 
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DM’s  Order 

Model's  Order 

Overall  Scores  From  Model 

M-1 

M-5 

0.35S 

M-5 

M-1 

0.335 

M-4 

M-4 

0.303 

M-3 

M-3 

0.231 

M-2 

M-2 

0.223 

Table  25:  Solutions  for  Problem-1 


The  DM  and  model  ordered  the  alternatives  according  to  the  related  data  on  Table 
24.  Table  25  shows  the  evaluations  by  the  two  sources.  Since  there  is  a  difference 
between  the  two  solutions,  this  model  may  not  be  valid.  However,  running  sensitivity 
analysis  with  the  software,  the  DM  was  able  to  see  the  reason  for  the  difference.  The 
following  figures  show  one  way  sensitivity  analysis  on  the  currency  and  wanted  readiness 
measures. 


Figure  16:  One  Way  Sensitivity  Analysis  for  Currency 
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Figure  17:  One  Way  Sensitivity  Anaiysis  for  Wanted  Readiness 


Using  the  results  from  the  sensitivity  analysis  for  currency  and  wanted  readiness, 
the  DM  is  asked  if  lowering  the  weight  of  currency  from  0.382  to  approximately  0.3  or 
increasing  the  weight  of  wanted  readiness  from  0.127  to  almost  0.2  is  reasonable.  The 
DM  did  not  want  to  change  the  weights  and  accepted  the  model  ordering  of  the 
alternatives. 

The  next  problem  is  similar  to  Problem- 1  to  be  sure  about  the  currency  and 
wanted  readiness  values  tradeoffs.  The  following  table  shows  the  random  inputs  for 
Problem-2. 
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Missious 

Current  Days 

Wanted  Days 

Currency  Left 

Wanted  Left 

M-1 

120 

30 

67 

0 

M-2 

90 

60 

3S 

8 

M-3 

90 

90 

39 

39 

M-4 

60 

60 

10 

10 

M-5 

60 

30 

11 

0 

Table  26:  Inputs  for  Problem-2 


The  DM  was  solicited  after  being  shown  Table  26.  Then  the  model  was  run  for 
these  inputs.  Other  inputs  were  selected  as  the  same  for  all  mission  types. 


DM’s  Order 

Model’s  Order 

Overall  Scores  From  Model 

M-5 

M-5 

0.63  6 

M-4 

M-4 

0.629 

M-2 

M-2 

0.350 

M-1 

M-1 

0.335 

M-3 

M-3 

0.244 

Table  27:  Solutions  for  Problem-2 


According  to  the  DM  and  the  model,  the  solutions  are  exactly  the  same  so  that  the 
tradeoff  between  currency  and  wanted  readiness  is  answered  correctly  by  the  model. 

Problem-3  was  prepared  to  see  the  priorities  considering  three  measures, 
currency,  wanted  readiness  and  personal  need. 


Missions 

Current  Days 

Wanted  Days 

Currency  Left 

Wanted  Left 

Personal  Need 

M-1 

120 

37 

88 

5 

4 

M-2 

120 

53 

98 

31 

1 

M-3 

120 

36 

118 

34 

4 

M-4 

120 

37 

67 

0 

4 

M-5 

120 

38 

96 

14 

5 

Table  28:  Inputs  for  Problem-3 
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DM’s  Order 

Model's  Order 

Overall  Scores  From  Model 

M-4 

M-4 

0.341 

M-1 

M-1 

0.32S 

M-5 

M-5 

0.2S4 

M-2 

M-2 

0.244 

M-3 

M-3 

0.216 

Table  29:  Solutions  for  Problem-3 


Due  to  the  increase  in  the  count  of  measures  to  be  considered,  the  DM  needed 
more  time  than  the  previous  problems.  However,  the  solutions  are  the  same  and  thus 
Table  29  validates  the  model  so  far. 

The  next  problem  tries  to  introduce  the  DM  to  Personal  Need  versus  Career.  It 
uses  consecutive  numbers  for  Currency  and  Wanted  Readiness  entries  in  order  to 
simplify  the  problem  for  the  DM.  The  other  eight  evaluation  measures  were  assumed  to 


be  constant  for  all  missions. 


Missions 

Currency  Left 

Wanted  Left 

Personal  Need 

Career 

M-1 

SS 

SS 

4 

1 

M-2 

ss 

S9 

1 

1 

M-3 

90 

90 

4 

1 

M-4 

91 

91 

4 

0 

M-5 

92 

92 

5 

2 

Table  30:  Inputs  for  Problem-4 

While  soliciting  the  DM  for  the  solution  to  Problem-4,  he  had  difficulty  deciding 
between  M-3  and  M-5.  The  following  table  verifies  the  confusion  and  the  difference 
between  the  DM’s  and  model’s  solution. 
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DM’s  Order 

Model's  Order 

Overall  Scores  From  Model 

M-3 

M-5 

0.302 

M-5 

M-3 

0.292 

M-1 

M-1 

0.2S7 

M-2 

M-2 

0.270 

M-4 

M-4 

0.220 

Table  31 :  Solutions  for  Problem-4 

As  far  as  overall  scores  concerned,  it  is  hard  to  decide  the  order  between  any 
alternatives  because  the  numbers  are  similar.  However,  sensitivity  analysis  can  be  done  in 
order  to  clear  up  the  confusion.  The  following  figures  reveal  the  sensitivity  of  results  on 
the  change  of  Personal  Need  or  Career  weights. 


Figure  18:  One  Way  Sensitivity  Analysis  for  Personal  Need 
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Figure  19:  Zoomed  View  for  Figure-18 


Figure  18  and  Figure  19  demonstrate  why  a  decision  between  M-1,  M-3  and  M-5 
is  difficult.  With  a  slight  change  in  weight  of  Personal  Need,  every  order  is  possible 
between  these  three  missions. 
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Figure  20:  One  Way  Sensitivity  Anaiysis  for  Career 

Figure  20  shows  the  effects  of  changes  on  Career’s  weight.  M-2  and  M-4  are 
always  the  bottom  two  but  the  others  are  again  hard  to  order.  Finally,  the  DM  accepted 
that  Career  is  more  important  than  Personal  Need  and  model’s  order  is  acceptable  in 
these  kinds  of  situations.  Nevertheless  running  sensitivity  analysis  is  crucial  when  scores 
are  so  close. 

The  last  problem  in  this  part  is  the  hardest  for  the  DM  to  solve  and  decide  on  the 
order  of  alternatives  because  Currency,  Wanted  Readiness,  Personal  Need  and  Career 
inputs  are  all  provided  randomly. 
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MissioDs 

Currency  Left 

Wanted  Left 

Personsil  Need 

Career 

M-1 

SS 

9 

2 

I 

M-2 

ss 

58 

3 

0 

M-3 

90 

32 

2 

1 

M-4 

91 

4 

3 

0 

M-5 

92 

9 

5 

0 

Table  32:  Inputs  for  Problem-5 


DM’s  Order 

Model’s  Order 

Overall  Scores  From  Model 

M-4 

M-4 

0.397 

M-1 

M-1 

0.335 

M-5 

M-5 

0.313 

M-2 

M-2 

0.306 

M-3 

M-3 

0.218 

Table  33:  Solutions  for  Problem-5 


According  to  the  DM,  the  last  three  are  obvious  but  deciding  the  first  two  rows 
needs  some  time.  However,  since  M-4  has  only  four  days  in  wanted  readiness,  the  weekly 
schedule  should  contain  this  assignment  for  the  purpose  of  the  readiness  value  and  the 
DM  and  model  agreed  in  their  orderings. 

4.2.2  Comparing  Assignments  of  Five  Different  Pilots 

The  other  class  of  problems  in  the  validation  part  creates  alternative  assignments 
that  contain  five  different  pilots  and  five  available  missions,  respectively.  This  part  aims 
to  show  more  complicated  problems  and  the  solutions  of  them  by  two  operators. 


Pilots 

Missions 

Wanted  Left 

Yearly  Flight 

Yearly  Presence 

P-1 

M-1 

99 

103 

171 

P-2 

M-2 

101 

17 

134 

P-3 

M-3 

118 

44 

130 

P-4 

M-4 

33 

108 

144 

P-5 

M-5 

77 

67 

151 

Table  34:  Inputs  for  Problem-6 
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Table  34  shows  the  randomized  inputs  for  the  Wanted  Readiness  and  Yearly 
Scheme  evaluation  measures.  In  this  problem,  the  model  compares  these  two  measures 
according  to  the  DM’s  preferences  and  decides  the  order  of  the  five  assignments.  Other 
needed  inputs  were  kept  constant  for  all  matches.  The  following  table  introduces  the 
results. 


DM’s  Order 

Model’s  Order 

Overall  Scores  From  Model 

P2-M2 

P2-^^2 

0.287 

P4-M4 

P4-M4 

0.282 

P3-M3 

P3-M3 

0.254 

P5-M5 

P5-M5 

0.222 

PI -Ml 

PI -Ml 

0.164 

Table  35:  Solutions  for  Problem-6 

With  respect  to  Table  35,  both  operators’  solutions  are  exactly  the  same.  These 


identical  results  support  validation  of  the  model. 


Pilots 

Missions 

Mouthly  Flight 

Montbh"  Presence 

Dutv'  Total 

Dutv'  !Mas 

P-1 

M-l 

0 

17 

14 

15 

P-2 

M-2 

2 

12 

3 

15 

P-3 

M-3 

1 

6 

4 

15 

P-4 

M-4 

4 

7 

15 

15 

P-5 

M-5 

9 

2 

12 

15 

Table  36:  Inputs  for  Problem-7 

Table  36  has  the  inputs  for  Problem-7  which  tests  the  comparison  between 
Monthly  Scheme  and  Duty  Equality.  All  inputs  were  acquired  randomly  in  their  limits 
and  measures  other  than  these  two  were  assumed  to  be  constant. 
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DM’s  Order 

Model's  Order 

Overall  Scores  From  Model 

P3-M3 

P3-M3 

0.286 

PI -Ml 

P4-M4 

0.264 

P4-M4 

PI -Ml 

0.256 

P2-M2 

P2-M2 

0.237 

P5-M5 

P5-M5 

0.203 

Table  37:  Solutions  for  Problem-7 

The  model’s  solution  is  a  little  different  than  the  DM’s  order  but  the  scores  of 
alternatives  ordered  differently  are  similar.  When  considering  alternatives  that  are 
confusing  to  the  DM,  the  DM  did  not  have  concern  with  monthly  presence.  However,  the 
structure  does  use  presences.  Finally,  the  DM  agreed  with  the  structure  and  admitted  that 
the  order  the  model  came  up  with  is  more  logical  based  on  the  monthly  presences. 
Because  Pilot-4  has  only  seven  days  left  this  month  to  fly,  he/she  needs  to  fly  more  than 
Pilot-1  who  has  seventeen  days  left.  Also,  Pilot-4  has  the  maximum  on  duty  numbers. 
Thus,  from  all  aspects  of  DM’s  preferences,  Pilot -4  should  fly  prior  to  Pilot- 1.  To  sum 
up,  VFT  works  and  these  situations  prove  just  that.  With  the  Value  Focused  Thinking 
method,  the  DM  defines  preferences  as  a  structure  first  and  then  the  structure  solves  the 
problem  according  to  the  DM’s  concerns.  Flight  scheduling  in  fighter  squadrons  is  a  huge 
problem  and  needs  structured  thinking  with  the  help  of  a  computer  that  can  consider  all 


entries  thoroughly. 


Pilots 

Missions 

Weekly  Flight 

Weekly  Presence 

Permission  Requests 

Past  Percentage 

p-i 

M-l 

3 

1 

0 

1 

P-2 

M-2 

0 

2 

0 

0.33 

P-3 

M-3 

2 

1 

1 

0 

P-4 

M-4 

0 

1 

0 

0.25 

P-5 

M-5 

2 

3 

0 

0 

Table  38:  Inputs  for  Problem-8 
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Problem-8  seeks  the  validation  for  trade-offs  between  Weekly  Scheme  and 
Permission  measures.  Table  38  shows  the  inputs  for  this  problem.  The  Permission 
Requests  column  tells  about  whether  the  respective  pilot  wants  permission  to  not  fly.  The 
Past  Percentage  column  calculates  the  percentage  of  permission  requests  positively 
answered  by  the  commander. 


DM’s  Order 

Model's  Order 

Overall  Scores  From  Model 

P3-M3 

P3-M3 

0.297 

P2-M2 

P2-M2 

0.286 

P4-M4 

P4-M4 

0.285 

PI -Ml 

PI -Ml 

0.276 

P5-M5 

P5-M5 

0.268 

Table  39:  Solutions  for  Problem-8 


Table  39  introduces  the  DM’s  and  model’s  solutions  for  Problem-8.  In  terms  of 
evaluating  the  trade-off  between  Weekly  Scheme  and  Permission,  there  seems  to  be  no 


difference  among  the  two  solutions. 


Pilots 

MissioDs 

Yearly  Plight 

Monthly  Flight 

Weekly  Flight 

Daily  Flight 

P-1 

M-1 

46 

5 

3 

0 

P-2 

M-2 

70 

9 

2 

0 

P-3 

M-3 

31 

3 

0 

0 

P-4 

M-4 

54 

5 

3 

0 

P-5 

M-5 

7 

5 

1 

0 

Table  40:  Inputs  for  Problem-9  Part-1 


Pilots 

Missions 

Yearly  Presence 

Monthly  Presence 

Weekly  Presence 

Daily  Presence 

P-1 

M-1 

170 

14 

1 

1 

P-2 

M-2 

183 

1 

4 

3 

P-3 

M-3 

139 

1 

5 

1 

P-4 

M-4 

146 

12 

1 

1 

P-5 

M-5 

114 

17 

1 

1 

Table  41 :  Inputs  for  Problem-9  Part-2 
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Problem-9  is  a  challenging  problem  and  examines  the  reactions  of  the  model  and 
the  DM  to  the  kind  of  problem  that  has  a  lot  of  variables  to  analyze.  Inputs  for  the 
Frequency  value  were  found  with  the  Excel  RandBetween  function  and  other  variables 
were  assumed  to  be  constant.  In  this  problem,  the  DM  and  model  have  to  compare 
priorities  between  the  Yearly  Scheme,  Monthly  Scheme,  Weekly  Wanted  and  Daily 
Wanted  evaluation  measures  and  decide  the  order  of  five  alternatives  randomly 
constructed.  Table  40  states  the  number  of  flights  as  yearly,  monthly,  weekly  and  daily 
for  five  pilots.  Furthermore,  Table  41  reveals  the  available  presences  for  these  pilots  in 
this  year,  month,  week  and  day. 


DM's  Order 

Model's  Order 

Overall  Scores  From  Model 

P5-M5 

P3-M3 

0.387 

P3-M3 

P5-M5 

0.371 

P4-M4 

P4-M4 

0.266 

PI -Ml 

PI -Ml 

0.260 

P2-M2 

P2-M2 

0.187 

Table  42:  Solutions  for  Problem-9 

According  to  the  solutions  shown  in  Table  42,  there  is  a  difference  between  the 
DM’s  and  the  model’s  orders.  The  DM  wants  to  schedule  Pilot-5  prior  to  Pilot-3  contrary 


to  the  model.  Before  changing  the  decision  model  we  first  review  the  structure. 


Weights 

0.170 

0.097 

0.020 

0.039 

Pilots 

Missions 

Yearly  Scheme 

Monthly  Scheme 

Weekly  Scheme 

Daily  Scheme 

P-3 

M-3 

0.640 

1.00 

0.60 

1 

P-5 

M-5 

0.991 

0.18 

1.00 

1 

Table  43:  Scores  for  Frequency  Evaluation  Measures  in  Problem-9 
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In  terms  of  daily  schemes,  there  is  no  difference  between  pilots  as  can  be  seen 
from  Table  43.  Considering  weights,  the  most  important  measure  is  Yearly  Scheme  and 
then  Monthly  Scheme.  The  differences  between  scores  show  that  the  monthly  flight  count 
of  Pilot-3  needs  to  be  boosted  immediately.  Also,  Pilot-3  has  only  one  day  in  that  month 
to  fly  but  Pilot-5  has  seventeen  available  days,  so  he/she  can  accomplish  the  monthly 
goal  more  easily.  After  the  explanation,  the  DM  did  not  want  to  change  the  weights  and 


kept  the  model’s  scores  and  rankings. 


Pilots 

Missions 

Currency  Left 

Wanted  Left 

Yearly  Mas 

Monthly  Mas 

p-i 

M-l 

55 

0 

83 

7 

P-2 

M-2 

58 

7 

83 

7 

P-3 

M-3 

81 

30 

83 

7 

P-4 

M-4 

71 

40 

83 

7 

P-5 

M-5 

73 

51 

83 

Table  44:  Inputs  for  Problem-10  Part-1 


Pilots 

Missions 

Yearly  Flight 

Monthly  Flight 

Weekly  Flight 

Daily  Flight 

p-i 

M-l 

77 

0 

1 

0 

p-2 

M-2 

78 

6 

0 

0 

P-3 

M-3 

8 

3 

3 

0 

P-4 

M-4 

24 

7 

1 

0 

P-5 

M-5 

83 

3 

1 

0 

Table  45:  Inputs  for  Problem-10  Part-2 


Pilots 

Missions 

Yearly  Presence 

Monthly  Presence 

Weekly  Presence 

Daily  Presence 

p-i 

M-l 

161 

17 

2 

3 

p-2 

M-2 

115 

19 

5 

2 

P-3 

M-3 

145 

IS 

4 

2 

P-4 

M-4 

116 

11 

4 

3 

P-5 

M-5 

113 

15 

3 

3 

Table  46:  Inputs  for  Problem-10  Part-3 


As  seen  on  Table  44,  Table  45  and  Table  46,  the  last  validation  problem  requires 
the  comparison  of  the  preferences  on  the  Readiness  and  Frequency  values  and  also  two 
measures  of  the  Motivation  value.  It  takes  into  account  eight  out  of  the  twelve  measures 


and  assumes  the  other  four  measures  are  unchanged.  The  following  table  shows  the 
results. 


DM’s  Order 

Model's  Order 

Overall  Scores  From  Model 

P3-M3 

P3-M3 

0.358 

P4-M4 

P4-M4 

0.323 

PI -Ml 

PI -Ml 

0.308 

P2-M2 

P2-M2 

0.288 

P5-M5 

P5-M5 

0.219 

Table  47:  Solutions  for  Problem-10 


According  to  the  orders  shown  in  Table  47,  there  is  no  difference  between  the 
solutions.  The  model  seems  accurate  even  in  difficult  and  complicated  problems  such  as 
Problem- 10  which  has  eight  variables. 

4.3  Summary 

This  chapter  discussed  verification  and  validation  of  the  research  model.  The  first 
part  verified  the  calculations  in  this  research  and  the  software  program  named  Hierarchy 
Builder  2.0  which  was  used  to  build  the  VFT  model  (Weir  J.  ,  2008).  The  second  part 
validated  the  model  via  comparing  solutions  created  by  the  DM  and  the  research  model  in 
order  to  solve  sample  assignment  problems.  The  last  chapter  presents  conclusions  and 
recommendations  about  this  research. 
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5.  CONCLUSIONS 


In  this  chapter,  the  summary  of  research,  conclusions  and  future  research 
recommendations  are  presented. 

5.1  Summary  of  the  Research 

Chapter  1  starts  with  defining  the  general  problem  and  continues  by  stating  the 
specific  problem  of  this  research.  The  general  problem  is  the  assignment  problem  which 
is  one  of  the  main  problems  in  Operations  Research.  The  specific  problem  is  flight 
scheduling  in  fighter  squadrons.  After  revealing  the  problem,  the  scope  of  the  research  is 
mentioned  as  the  first  two  steps  of  the  process  of  modeling  the  scheduler.  Considering  the 
scope  of  this  research,  an  evaluation  model  using  VFT  is  built  and  a  helpful  program  is 
established  to  aid  the  scheduler  in  manually  building  schedules. 

The  next  chapter  of  this  research  reviews  recent  research  about  both  the  general 
problem  and  the  specific  problem.  After  reviewing  previous  works.  Chapter  2  tries  to 
make  clear  why  the  assignment  problem  or  flight  scheduling  problem  in  fighter 
squadrons  can  be  approached  using  decision  analysis  methods.  Thus,  decision  analysis 
methods  are  discussed  in  this  chapter.  The  last  part  of  Chapter  2  reviews  the  ten  step 
method  of  VFT  which  is  implemented  in  Chapter  3. 

Chapter  3  begins  with  general  information  about  flight  scheduling  in  fighter 
squadrons.  The  first  part  of  this  chapter  defines  inputs  and  outputs  of  the  specific  problem 
of  this  research.  The  second  part  of  Chapter  3  contains  the  first  seven  steps  of  VFT. 

Verification  and  validation  of  the  decision  model  created  with  the  help  of  the 
DM’s  preferences  are  introduced  in  Chapter  4.  A  sample  alternative  assignment  is 
evaluated  manually  and  by  the  model.  Furthermore,  ten  groups  of  alternatives  are 
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evaluated  and  ordered  by  the  DM  and  the  decision  model  in  order  to  compare  the 
solutions  and  validate  the  model. 

In  the  next  part  of  this  chapter,  analysis  results  are  explained  and  objectives  of  the 
research  and  accomplishments  are  discussed.  Finally,  future  recommendations  are  stated 
in  the  last  part  of  this  chapter. 

5.2  Conclusions 

Verification  of  the  model  as  discussed  in  Chapter  4  has  two  parts.  The  first  part 
considers  the  Hierarchy  Builder  2.0  software  program.  In  order  to  see  whether  the 
program  is  accurate,  one  sample  calculation  for  a  score  of  an  evaluation  measure  was 
made  by  hand  to  compare  results  with  the  model’s  results.  Nine  inputs  were  evaluated 
and  compared  with  model’s  results  and  the  average  difference  between  scores  was  too 
small  and  only  changes  the  scores  in  the  fifth  decimal  place.  As  far  as  this  research’s 
specific  problem  is  concerned,  this  type  of  accuracy  is  fine  and  the  model  is  accepted  as 
verified.  The  second  part  for  verification  focuses  on  the  changes  of  this  research  to  the 
software  program  for  the  purpose  of  accelerating  it.  Embellishments  made  by  this 
research  include  evaluating  the  scores  of  all  evaluation  measures  concerning  assignments 
immediately  while  entering  the  needed  inputs  of  the  alternatives.  Besides  this,  the  overall 
scores  are  calculated  immediately  for  all  alternatives  by  these  additions.  Ten  random 
alternatives  were  created  and  evaluated  by  Hierarchy  Builder  2.0  and  by  this  research’s 
model.  Because  the  software  program  uses  big  numbers  to  simulate  infinity,  there  were 
minor  differences  on  some  evaluation  measure  scores.  When  using  linear  functions  as 
SDVF  of  an  evaluation  measure,  the  program  gives  an  approximate  value  as  an  output. 
However,  the  research’s  model  uses  the  ratios  as  outputs  directly.  That  means  it  is  not 
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approximating  infinity  by  big  numbers  as  the  software  does  and  gives  exact  values  for  the 
corresponding  evaluation  measures.  Finally,  the  average  difference  between  overall 
scores  was  calculated.  It  is  not  a  significant  number  with  respect  to  this  research’s 
objectives. 

In  order  to  validate  the  decision  model,  ten  sample  groups  of  five  alternative 
assignments  are  created  by  randomizing  the  needed  inputs.  After  creating  the  groups  of 
alternatives,  the  DM  ordered  them  in  accordance  with  his  preferences.  The  value  model 
ordered  the  same  groups  of  sample  alternatives.  Then  the  validation  part  continues  with 
comparing  the  results  and  commenting  on  the  differences.  In  general,  there  were  only 
four  different  ordered  groups  out  of  the  ten  groups  and  these  differences  were  based  on 
only  two  assignments  in  these  different  ordered  groups.  Analyzing  the  differences,  some 
important  findings  are  discussed  here.  The  first  reason  that  causes  variation  between  the 
two  orderings  is  the  problem  of  the  DM’s  calibration.  The  decision  model  was  built  with 
the  inputs  of  the  DM  and  the  decision  process  was  structured  for  future  problems. 
However,  the  DM  still  thinks  in  an  unstructured  way.  He  thinks  about  one  evaluation 
measure  at  a  time  or  can  think  about  two  measures  and  decides  only  using  them.  Because 
the  DM  does  not  think  in  a  structured  way,  this  research  aims  to  structure  the  DM’s 
values  and  preferences.  The  validation  part  shows  that  the  DM’s  weakness  continues.  We 
need  to  calibrate  the  DM  first  and  then  solicit  him  again  after  this  procedure.  For  almost 
all  sample  groups,  this  process  was  applied  and  the  DM  accepted  the  solution  of  the 
model  instead  of  his  solution.  So  this  finding  validates  the  model  due  to  the  objectives  of 
this  research.  The  second  important  finding  of  the  validation  part  was  seen  after  the 
different  ordered  alternatives’  scores  are  examined.  The  differences  between  scores  were 
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too  small  and  thus  the  DM  could  choose  any  of  them  after  looking  at  the  big  picture  the 
model  provides.  Sensitivity  analysis  was  run  in  order  to  make  the  DM  think  about  the 
weights.  However,  the  DM  did  not  want  to  change  the  weights  because  the  needed  range 
of  change  was  not  logical  according  to  his  views.  This  finding  also  validates  the  model 
because  this  model  was  created  to  show  a  big  picture  and  to  help  the  DM  to  see  the 
priorities  between  assignments.  It  is  not  for  making  a  decision  for  the  best  assignment. 
After  getting  the  big  picture,  the  DM  can  choose  any  alternatives  by  his  experience  and 
make  the  schedule.  The  other  important  result  of  validation  was  about  time.  Some  of  the 
validation  samples  created  kept  almost  ten  variables  constant  but  some  of  them  used 
more  than  five  variables  and  entered  random  inputs.  The  DM  started  to  consume  a  large 
amount  of  time  thinking  about  those  problems  that  have  more  than  four  variables.  Even 
with  two  variables,  he  doesn’t  have  structured  thinking,  so  we  cannot  expect  accuracy  on 
bigger  problems.  Nevertheless,  the  model  does  not  consume  much  time  to  get  the 
evaluation  of  needed  alternatives  or  order  them.  This  fact  proves  the  model  useful  with 
regard  to  one  of  this  research’s  objectives  defined  in  Chapter  1.  This  goal  is  to  reduce  the 
total  time  it  takes  a  scheduler  to  do  his/her  job.  Using  the  model  instead  of  starting  from 
scratch  is  introducing  an  invaluable  time  advantage. 

In  Chapter  I,  four  objectives  are  stated  along  with  the  research  question.  The  first 
objective  is  providing  simplicity,  flexibility  and  structured  thinking  to  the  assignment 
problem.  Using  user  friendly  software  programs  such  as  Excel  and  Hierarchy  Builder  2.0 
simplifies  the  work  the  DM  does.  He/she  can  easily  add  some  steps  to  this  model  to  make 
a  more  robust  model  or  remove  some  parts  from  the  model  to  get  rid  of  unnecessary  or 
old  measures.  The  DM  can  easily  change  weights  or  values  because  the  programs  are 
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flexible.  Furthermore,  VFT  starts  with  defining  objectives  and  values;  then  it  continues 
with  structuring  a  decision  model  to  recommend  decisions  according  to  defined 
preferences. 

The  second  objective  achieved  is  showing  a  big  picture  of  the  current  state  of  the 
problem  to  the  decision  makers.  After  the  scoring  alternatives  step,  deterministic  analysis 
and  sensitivity  analysis  help  build  the  big  picture.  The  main  spreadsheet  for  the  overall 
score  gives  the  biggest  picture  for  the  problem.  However,  other  spreadsheets  are  obtained 
by  using  filters  in  Excel  in  order  to  see  a  specific  pilot’s  current  status  or  a  specific 
mission’s  assignment  scores.  The  DM  can  see  all  matches  at  the  same  time  or  he/she  can 
filter  the  assignments  and  can  acquire  necessary  assignments  to  be  able  to  think  about 
them  thoroughly.  He/she  can  choose  several  matches  and  then  run  a  sensitivity  analysis  to 
see  the  effects  of  weights.  These  abilities  prove  that  the  second  objective  was  achieved. 

The  next  objective  is  changing  subjective  decisions  to  objective  ones.  As  is 
written  in  Chapter  3,  the  flight  scheduling  process  in  fighter  squadrons  is  not  objective. 
There  are  several  decision  makers  and  subject  matter  experts  that  have  an  effect  on 
schedules.  They  have  different  objectives  and  preferences  and  the  Air  Force  also  has 
important  objectives  about  flying  missions.  The  VFT  method  constructs  an  objective 
model  and  the  decisions  made  with  the  help  of  this  model  will  be  repeatable  and 
consistent.  The  DM  may  decide  with  bias  but  the  model  does  not.  The  DMs  and  the 
SMEs  can  answer  the  same  problem  differently  but  the  model  will  be  consistent  and 
solve  the  same  problem  and  find  the  same  solution  every  time. 

The  last  objective  mentioned  in  Chapter  1  is  to  reduce  the  total  time  it  takes  a 
scheduler  to  do  their  job.  The  model  certainly  decreases  the  time  consumed  because  it 
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combines  all  evaluation  measures  into  only  one  overall  score.  Hence,  the  DM  can  use  the 
overall  scores  to  consider  the  assignments’  values  and  decide  in  less  time.  The  model  not 
only  decreases  the  amount  of  time  the  scheduler  consumes  but  also  increases  the  number 
of  variables  the  scheduler  takes  into  account.  The  DM  considers  only  two  or  three  of  the 
evaluation  measures  even  if  there  are  more  than  three  measures.  Using  the  decision 
model,  a  large  number  of  variables  can  be  evaluated  in  a  short  time  and  concisely. 
Furthermore,  the  DM  chooses  several  important  assignments  to  start  with  because  the 
problem  has  approximately  1500  assignment  alternatives.  He/she  thinks  like  that  because 
considering  all  of  the  alternatives  is  not  logical  due  to  time  limits.  Nevertheless,  a 
computer  model  can  do  this  work  in  a  small  amount  of  time  and  give  the  values  for  1500 
assignments  in  almost  one  second.  The  only  job  for  the  SME  is  to  create  a  feasible 
sequence  using  these  scores.  Finally,  this  model  reduces  the  total  time  for  the  scheduling 
process  and  makes  it  more  robust  because  it  does  not  skip  any  alternative. 

5.3  Future  Recommendations 

Future  researchers  should  focus  on  other  specific  assignment  problems  and  use 
this  approach  in  order  to  acquire  same  advantages  and  provide  the  same  objectives.  The 
VFT  method  can  be  used  in  all  types  of  assignment  problems  to  construct  a  decision 
model  and  help  DMs  save  time. 

In  this  research,  flight  scheduling  in  fighter  squadrons  is  held  as  an  assignment 
problem  to  show  how  to  approach  the  assignment  problem  with  a  different  method.  As 
described  in  Chapter  1,  modeling  the  scheduler  has  three  steps  and  this  research  tries  to 
solve  the  first  two  steps.  It  builds  the  decision  analysis  model  for  evaluating  the 
alternatives  and  makes  spreadsheets  to  aid  the  scheduler  in  manually  building  schedules. 
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The  third  step  is  left  for  future  research.  This  step  will  automate  the  process  of  pilot- 
mission  assignment  with  the  help  of  defined  values  and  objectives.  This  step  can  be 
approached  by  a  heuristic  method  like  GRASP  or  a  math  model.  The  advantage  of  a 
heuristic  method  is  that  it  is  less  time  consuming  but  it  does  not  guarantee  the  best 
solution.  The  Math  method  can  conclude  with  the  best  results,  but  it  may  take  a  long  time 
which  is  so  valuable  for  pilots. 

Future  research  may  also  want  to  simulate  a  squadron’s  life  throughout  the  flight 
year.  This  simulation  research  can  use  the  decision  model  created  by  this  research  in 
order  to  obtain  evaluation  of  the  assignment  alternatives,  and  it  can  use  random  numbers 
of  pilots  or  random  numbers  of  aircrafts  to  get  a  good  solution.  Furthermore,  this 
simulation  research  could  use  different  numbers  of  instructors  or  leaders  to  see  how  many 
of  them  are  actually  needed  for  fighter  squadrons.  These  numbers  can  be  an  input  for  Air 
Force  aircraft  projects  and  personal  assignments. 
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APPENDIX  A:  QUAD  CHART 
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APPENDIX  B:  VALUE  FUNCTIONS  AND  FORMULAS 


In  this  section  twelve  value  functions,  related  formulas  and  needed  data  will  be 


shown. 


Currencv 


Bjmaiiiiiig'CnrKnt  Pars_ j 


Figure  21 :  SDVF  for  Currency 


TODAY 

1/3/2011 

MISSIONS 

LAST  FLIGHT  DATE 

CURRENT  DAYS 

CURRENCY 

M-1 

12/13/2010 

120 

99 

M-13 

12/14/2010 

110 

90 

M-23 

12/15/2010 

100 

SI 

M-42 

12/16/2010 

90 

72 

M-50 

12/17/2010 

75 

5g 

Table  48:  Spreadsheet  for  Needed  Data  of  Currency  Scores  Calculations 
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TODAY 

Input 

MISSIONS 

LAST  FLIGHT  DATE 

CURRENT  DAYS 

Input 

Input 

Input 

CURRENCY 

Current  Days- (Today- Last  Flight  Date) 

Table  49:  Formulas  and  Descriptions  of  Entries  for  Currency  Spreadsheet 


Wanted  Readiness 


0  20  40  eo  BO  lt»  120 
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Figure  22:  SDVF  for  Wanted  Readiness 


TODAY 

1/3/2011 

MISSIONS 

LAST  FLIGHT  DATE 

WANTED  DAYS 

WANTED 

M-1 

12/13/2010 

100 

79 

M-13 

12/14/2010 

90 

70 

M-23 

12/15/2010 

SO 

61 

M-42 

12/16/2010 

70 

52 

M-50 

12/17/2010 

60 

43 

Table  50:  Spreadsheet  for  Needed  Data  of  Wanted  Readiness  Scores  Calculations 
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value 


TODAY 

Input 

MISSIONS 

LAST  FLIGHT  DATE 

WANTED  DAYS 

Input 

Input 

Input 

WANTED 

Wanted  Days-jToday-Last  Flight  Date) 

Table  51 :  Formulas  and  Descriptions  of  Entries  for  Wanted  Readiness 


Yearly  Scheme 


Yearly  Rada  j 

Figure  23:  SDVF  for  Yearly  Scheme 


YEARLY  FLIGHT 

YEARLY  SCHEME 

YEARLY  PRESENCE 

IS 

120 

173 

YEARLY  RATIO 

0.5S3S150ZS 

Table  52:  Spreadsheet  for  Needed  Data  of  Yearly  Scheme  Scores  Calculations 
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value 


YEARLY  FLIGHT 

YEARLY  SCHEME 

YEARLY  PRESENCE 

Input 

Input 

Input 

YEARLY  RATIO 


IF{{Yearly  Schenne-YEarly  Flight) /Yearly 
Presence  >OJF{{Yearly  Seheine-Yearly  Flight)/Yearly 
Presence<lj  {Yearly  Scheme-Yearly  Flight)/Yearly 
Presenee,l],0} 


Table  53:  Formulas  and  Descriptions  of  Entries  for  Yearly  Scheme 


Monthly  Scheme 


Figure  24:  SDVF  for  Monthly  Scheme 


MONTHLY  FLIGHT 

MONTHLY  SCHEME 

MONTHLY  PRESENCE 

3 

S 

21 

MONTHLY  RATIO 

O.Z3SOS5Z3S 

Table  54:  Spreadsheet  for  Needed  Data  of  Monthly  Scheme  Scores  Calculations 
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Valye 


MONTHLY  FLIGHT 

MONTHLY  SCHEME 

MONTHLY  PRESENCE 

Input 

Input 

Input 

MONTHLY  RATIO 


IF{{Monthly  Scheme-Monthly  FlightJ/Monthly 
Presence  >{),IF{{Monthly  Scheme-Monthly  FlightJ/Monthly 
Presence<l,{Monthly  Scheme-Monthly  FlightJ/Monthly 
Presence,l],0) 


Table  55:  Formulas  and  Descriptions  of  Entries  for  Monthiy  Scheme 


Weekly  Wanted 


Weekty  Ratio 


Figure  25:  SDVF  for  Weekiy  Wanted 


WEEKLY  FLIGHT 

WEEKLY  WANTED 

WEEKLY  PRESENCE 

2 

3 

5 

WEEKLY  RATIO 

0.2 

Tabie  56:  Spreadsheet  for  Needed  Data  of  Weekiy  Wanted  Scores  Caicuiations 


102 


WEEKLY  FLIGHT 

WEEKLY  WANTED 

WEEKLY  PRESENCE 

input 

Input 

Input 

WEEKLY  RATIO 


IF{[We0kly  Wants  d-Weekly  Flight}/W66kly  Pr6S6nc6>0'JF{{W66kly 
Wants  d-Weekly  Flight}/Wsskly  Prsssncs<lj{W66kly  Wants  d-Wsskly 
Flight}/W66kly  Prss6nes,l),0) 


Table  57:  Formulas  and  Descriptions  of  Entries  for  Weekiy  Wanted 


Da  ily  Wanted 


Dai]  Y  Ratio_ J 

Figure  26:  SDVF  for  Daiiy  Wanted 


DAILY  FLIGHT 

DAILY  WANTED 

DAILY  PRESENCE 

0 

1 

3 

DAILY  RATIO 

0.333333333 

Tabie  58:  Spreadsheet  for  Needed  Data  of  Daiiy  Wanted  Scores  Caicuiations 
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DAILY  FLIGHT 

DAILY  WANTED 

DAILY  PRESENCE 

Input 

Input 

Input 

DAILY  RATIO 


IF{{Daily  Wanted-Daily  FlightJ/Daily  PrGsence:=OJF{{Daily  Wanted- 
Daily  FlightJ/Daily  PrGSGncG<l,{Daily  Wanted-Daily  FlightJ/Daily 

PresencG,lJ,0'J 


Table  59:  Formulas  and  Descriptions  of  Entries  for  Daily  Wanted 


Career 

1.00  n - 

o.so - 

o.ao - 

0.70 - 

0.60 - 

I  0.0 - 1 

0.40 - 

0.30 - 

0.20 - 

0.10 - 


0 

1 

2 

■Category 

0.00 

0.75 

1.00 

Figure  27:  SDVF  for  Career 


MISSIONS 

CAT-III 

CAT-II 

CAT-1 

BEGINNER 

2  SHIP  LEADER 

4  SHIP  LEADER 

INSTRUCTOR 

REFRESHMENT 

TOTAL 

M-1 

0 

0 

0 

0 

0 

0 

0 

0 

0 

M-13 

1 

0 

0 

1 

0 

0 

0 

0 

2 

M-23 

0 

0 

0 

1 

0 

0 

0 

0 

1 

M-42 

0 

1 

0 

0 

1 

0 

0 

0 

2 

M-50 

0 

0 

0 

0 

0 

0 

1 

0 

1 

Table  60:  Spreadsheet  for  Needed  Data  of  Career  Scores  Calculations 
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MISSIONS 

CAT-III 

CAT-II 

CAT-I 

BEGINNER 

2  SHIP  LEADER 

4  SHIP  LEADER 

INSTRUCTOR 

REFRESHMENT 

Input 

Input 

Input 

Input 

Input 

Input 

Input 

Input 

Input 

TOTAL 

(CAT-lll)+(CAT-ll)+(CAT-l)+{BEGINNER)+(2  SHIP  LEADER)+{4  SHIP  LEADER)+(INSTRUCTOR}+{REFRESHMENT) 

Table  61 :  Formulas  and  Descriptions  of  Entries  for  Career 


Figure  28:  SDVF  for  Personai  Need 


MISSIONS 

PERSONAL 

M-1 

5 

M-13 

4 

M-23 

3 

M-42 

2 

M-50 

1 

Tabie  62:  Spreadsheet  for  Needed  Data  of  Personai  Need 
Scores  Caicuiations 
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MISSIONS 

PERSONAL 

Input 

Input 

Input 

Input 

Input 

Input 

Input 

Input 

Input 

Input 

Table  63:  Formulas  and  Descriptions  of  Entries  for  Personai  Need 


Permhssiori 

1 

0.9 
0.3 

0.7 
0-6 


O.fl 

0.3 

0-2 

0-1 

Q 


0  0.1 

D.2  0.3  0.4  O.b  O.E  0.7 

Past  RfsiiJt  Perc^ntace 

o.a  0.0  1 

Figure  29:  SDVF  for  Permission 

WANT  TO  FLY? 

PAST  REQUESTS 

PAST  RESULTS 

PAST  PERCENTAGE 

0 

5 

4 

0.8 

PERMISSION 

0.8 

Tabie  64:  Spreadsheet  for  Needed  Data  of  Permission  Scores  Caicuiations 


WANT  TO  FLY? 

PAST  REQUESTS 

PAST  RESULTS 

PAST  PERCENTAGE 

Input 

Input 

Input 

Past  Results/Past  Requests 

PERMISSION 

IF(Wantto  Fly=0,Past  Percentage,!) 

Tabie  65:  Formuias  and  Descriptions  of  Entries  for  Permission 
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value 


Yearly  Equality 


Yearly nighlBiJgierejifp^  at  Same  T.fvfl  J 

Figure  30:  SDVF  for  Yearly  Equality 


YEARLY  FLIGHT 

YEARLY  MAX 

19 

35 

YEARLY  DIFF 

16 

Table  66:  Spreadsheet  for  Needed  Data  of  Yearly  Equality 
Scores  Calculations 


YEARLY  FLIGHT  YEARLY  MAX 
_ Input _ Input 

_ YEARLY  DIFF _ 

IFfYearly  Max-Yearly 
Flight>30,30>Yearly  Max- Yearly 
Flight} 

Table  67:  Formulas  and  Descriptions  of  Entries  for  Yearly  Equality 
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\^hie 


Monthly  Equality 


Monthlr  Flight  Differences  at  Same  Level ^ 


Figure  31 :  SDVF  for  Monthly  Equality 


MONTHLY  FLIGHT 

MONTHLY  MAX 

3 

5 

MONTHLY  DIFF 

2 

Table  68:  Spreadsheet  for  Needed  Data  of  Monthly  Equality 
Scores  Calculations 


MONTHLY  FLIGHT  MONTHLY  MAX 

Input  Input 

_ MONTHLY  DIFF _ 

IF{Monthly  Max-Monthly 
F I ight>S,S, Monthly  Max- Monthly 
Flight} 

Table  69:  Formulas  and  Descriptions  of  Entries  for  Monthly  Equality 
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V^lue 


Figure  32:  SDVF  for  Duty  Equality 


DUTYl 

DUTY  2 

TOTAL 

TOTAL  MAX 

1 

2 

3 

5 

DIFFERENCE 

2 

Table  70:  Spreadsheet  for  Needed  Data  of  Duty  Equality 
Scores  Calculations 


DUTYl 

DUTY  2 

TOTAL 

TOTAL  MAX 

Input 

Input 

Dutyl+DutyZ 

Input 

DIFFERENCE 

IF(Total  Max-Total>10AO,Total  Max-Total} 

Table  71 :  Formulas  and  Descriptions  of  Entries  for  Duty  Equality 
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